From 448232c8789350c8fb071902d33a5c5f07d2b82c Mon Sep 17 00:00:00 2001 From: Yevgeny Pats Date: Tue, 17 Jan 2023 08:17:25 -0500 Subject: [PATCH 1/5] feat: Dynamic tables and introduce proto versioning (#610) This introduce WithDynamicTables for source plugins to give an option for plugins to generate tables on the go, this can be useful in GA analytics, Salesforce and others where you can only get the schema and tables in runtime. Give WithDynamicTables is actually a breaking change and I didn't want to turn our code into spagetti that tries to understand in runtime which version is the client I took the opportunity to fix our protobuf structures and introduce a way for creating new versions for server in a breaking way while also serving older versions of the server so it wont affect user experience but will give us a better way to write and maintain different versions and eventually deprecate older versions. cc @erezrokah as I know you suggested exactly that a while back but it was too early for me after the initial refactor :) and wasn't sure what are the requirements yet but here we go! --- Makefile | 3 +- buf.yaml | 12 + clients/{ => destination/v0}/constants.go | 2 +- clients/{ => destination/v0}/destination.go | 88 +- .../v0}/destination_terminate.go | 4 +- .../v0}/destination_terminate_windows.go | 4 +- .../{ => destination/v0}/destination_test.go | 6 +- clients/errors.go | 22 - clients/{ => source/v0}/source.go | 92 +- clients/{ => source/v0}/source_terminate.go | 4 +- .../v0}/source_terminate_windows.go | 4 +- clients/{ => source/v0}/source_test.go | 4 +- clients/source/v1/source.go | 303 ++++ clients/source/v1/source_terminate.go | 30 + clients/source/v1/source_terminate_windows.go | 20 + .../pb => cloudquery/base/v0}/base.proto | 5 +- .../destination/v0}/destination.proto | 15 +- .../pb => cloudquery/source/v0}/source.proto | 10 +- cloudquery/source/v1/source.proto | 77 + {clients => internal/logging}/log_reader.go | 14 +- .../logging}/log_reader_test.go | 9 +- {clients => internal/logging}/logging.go | 4 +- internal/pb/{ => base/v0}/base.pb.go | 186 +-- internal/pb/destination/v0/constants.go | 3 + .../pb/{ => destination/v0}/destination.pb.go | 416 +++--- .../v0}/destination_grpc.pb.go | 63 +- .../{servers => pb/source/v0}/constants.go | 2 +- internal/pb/{ => source/v0}/source.pb.go | 265 ++-- internal/pb/{ => source/v0}/source_grpc.pb.go | 53 +- internal/pb/source/v1/constants.go | 3 + internal/pb/source/v1/source.pb.go | 1325 +++++++++++++++++ internal/pb/source/v1/source_grpc.pb.go | 363 +++++ {clients => internal/random}/random.go | 4 +- .../{ => destination/v0}/destinations.go | 19 +- internal/servers/{ => source/v0}/source.go | 54 +- internal/servers/source/v1/source.go | 150 ++ plugins/destination/plugin.go | 8 +- plugins/source/benchmark_test.go | 4 +- plugins/source/options.go | 18 + plugins/source/plugin.go | 144 +- plugins/source/plugin_test.go | 5 +- plugins/source/testing.go | 6 +- {clients => registry}/download.go | 6 +- serve/destination.go | 10 +- ...ination_test.go => destination_v0_test.go} | 4 +- serve/source.go | 16 +- serve/source_v0_test.go | 156 ++ serve/{source_test.go => source_v1_test.go} | 17 +- 48 files changed, 3306 insertions(+), 726 deletions(-) create mode 100644 buf.yaml rename clients/{ => destination/v0}/constants.go (72%) rename clients/{ => destination/v0}/destination.go (74%) rename clients/{ => destination/v0}/destination_terminate.go (89%) rename clients/{ => destination/v0}/destination_terminate_windows.go (87%) rename clients/{ => destination/v0}/destination_test.go (91%) delete mode 100644 clients/errors.go rename clients/{ => source/v0}/source.go (74%) rename clients/{ => source/v0}/source_terminate.go (89%) rename clients/{ => source/v0}/source_terminate_windows.go (88%) rename clients/{ => source/v0}/source_test.go (91%) create mode 100644 clients/source/v1/source.go create mode 100644 clients/source/v1/source_terminate.go create mode 100644 clients/source/v1/source_terminate_windows.go rename {internal/pb => cloudquery/base/v0}/base.proto (86%) rename {internal/pb => cloudquery/destination/v0}/destination.proto (93%) rename {internal/pb => cloudquery/source/v0}/source.proto (91%) create mode 100644 cloudquery/source/v1/source.proto rename {clients => internal/logging}/log_reader.go (85%) rename {clients => internal/logging}/log_reader_test.go (95%) rename {clients => internal/logging}/logging.go (84%) rename internal/pb/{ => base/v0}/base.pb.go (69%) create mode 100644 internal/pb/destination/v0/constants.go rename internal/pb/{ => destination/v0}/destination.pb.go (59%) rename internal/pb/{ => destination/v0}/destination_grpc.pb.go (87%) rename internal/{servers => pb/source/v0}/constants.go (75%) rename internal/pb/{ => source/v0}/source.pb.go (77%) rename internal/pb/{ => source/v0}/source_grpc.pb.go (87%) create mode 100644 internal/pb/source/v1/constants.go create mode 100644 internal/pb/source/v1/source.pb.go create mode 100644 internal/pb/source/v1/source_grpc.pb.go rename {clients => internal/random}/random.go (89%) rename internal/servers/{ => destination/v0}/destinations.go (87%) rename internal/servers/{ => source/v0}/source.go (67%) create mode 100644 internal/servers/source/v1/source.go create mode 100644 plugins/source/options.go rename {clients => registry}/download.go (97%) rename serve/{destination_test.go => destination_v0_test.go} (94%) create mode 100644 serve/source_v0_test.go rename serve/{source_test.go => source_v1_test.go} (89%) diff --git a/Makefile b/Makefile index 9132f07fa7..5605f02a40 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,8 @@ lint: .PHONY: gen-proto gen-proto: - protoc --proto_path=. --go_out . --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative internal/pb/base.proto internal/pb/source.proto internal/pb/destination.proto + protoc --proto_path=. --go_out . --go_opt=module="github.com/cloudquery/plugin-sdk" --go-grpc_out=. --go-grpc_opt=module="github.com/cloudquery/plugin-sdk" cloudquery/base/v0/base.proto cloudquery/destination/v0/destination.proto cloudquery/source/v0/source.proto + protoc --proto_path=. --go_out . --go_opt=module="github.com/cloudquery/plugin-sdk" --go-grpc_out=. --go-grpc_opt=module="github.com/cloudquery/plugin-sdk" cloudquery/source/v1/source.proto .PHONY: benchmark benchmark: diff --git a/buf.yaml b/buf.yaml new file mode 100644 index 0000000000..b348cd312c --- /dev/null +++ b/buf.yaml @@ -0,0 +1,12 @@ +version: v1 +breaking: + use: + - FILE +lint: + use: + - BASIC + ignore: + # We are ignoring those as this is an old version and we are not doing any changes here anymore + - cloudquery/destination/v0/destination.proto + - cloudquery/source/v0/source.proto + - cloudquery/base/v0/base.proto diff --git a/clients/constants.go b/clients/destination/v0/constants.go similarity index 72% rename from clients/constants.go rename to clients/destination/v0/constants.go index ab8529dc4c..72bf65000e 100644 --- a/clients/constants.go +++ b/clients/destination/v0/constants.go @@ -1,4 +1,4 @@ -package clients +package destination const ( maxMsgSize = 100 * 1024 * 1024 // 100 MiB diff --git a/clients/destination.go b/clients/destination/v0/destination.go similarity index 74% rename from clients/destination.go rename to clients/destination/v0/destination.go index 9822971220..b37571a9d9 100644 --- a/clients/destination.go +++ b/clients/destination/v0/destination.go @@ -1,4 +1,4 @@ -package clients +package destination import ( "context" @@ -14,8 +14,12 @@ import ( "sync" "time" - "github.com/cloudquery/plugin-sdk/internal/pb" + "github.com/cloudquery/plugin-sdk/internal/logging" + pbBase "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" + pb "github.com/cloudquery/plugin-sdk/internal/pb/destination/v0" + "github.com/cloudquery/plugin-sdk/internal/random" "github.com/cloudquery/plugin-sdk/plugins/destination" + "github.com/cloudquery/plugin-sdk/registry" "github.com/cloudquery/plugin-sdk/schema" "github.com/cloudquery/plugin-sdk/specs" "github.com/rs/zerolog" @@ -26,7 +30,7 @@ import ( "google.golang.org/protobuf/types/known/timestamppb" ) -type DestinationClient struct { +type Client struct { pbClient pb.DestinationClient directory string cmd *exec.Cmd @@ -38,43 +42,43 @@ type DestinationClient struct { wg *sync.WaitGroup } -type DestinationClientOption func(*DestinationClient) +type ClientOption func(*Client) -func WithDestinationLogger(logger zerolog.Logger) func(*DestinationClient) { - return func(c *DestinationClient) { +func WithDestinationLogger(logger zerolog.Logger) func(*Client) { + return func(c *Client) { c.logger = logger } } -func WithDestinationDirectory(directory string) func(*DestinationClient) { - return func(c *DestinationClient) { +func WithDestinationDirectory(directory string) func(*Client) { + return func(c *Client) { c.directory = directory } } -func WithDestinationGrpcConn(userConn *grpc.ClientConn) func(*DestinationClient) { - return func(c *DestinationClient) { +func WithDestinationGrpcConn(userConn *grpc.ClientConn) func(*Client) { + return func(c *Client) { // we use a different variable here because we don't want to close a connection that wasn't created by us. c.userConn = userConn } } -func WithDestinationNoSentry() func(*DestinationClient) { - return func(c *DestinationClient) { +func WithDestinationNoSentry() func(*Client) { + return func(c *Client) { c.noSentry = true } } -func NewDestinationClient(ctx context.Context, registry specs.Registry, path string, version string, opts ...DestinationClientOption) (*DestinationClient, error) { +func NewClient(ctx context.Context, registrySpec specs.Registry, path string, version string, opts ...ClientOption) (*Client, error) { var err error - c := &DestinationClient{ - directory: DefaultDownloadDir, + c := &Client{ + directory: registry.DefaultDownloadDir, wg: &sync.WaitGroup{}, } for _, opt := range opts { opt(c) } - switch registry { + switch registrySpec { case specs.RegistryGrpc: if c.userConn == nil { c.conn, err = grpc.DialContext(ctx, path, @@ -102,25 +106,25 @@ func NewDestinationClient(ctx context.Context, registry specs.Registry, path str return nil, fmt.Errorf("invalid github plugin path: %s. format should be owner/repo", path) } org, name := pathSplit[0], pathSplit[1] - localPath := filepath.Join(c.directory, "plugins", string(PluginTypeDestination), org, name, version, "plugin") - localPath = withBinarySuffix(localPath) - if err := DownloadPluginFromGithub(ctx, localPath, org, name, version, PluginTypeDestination); err != nil { + localPath := filepath.Join(c.directory, "plugins", string(registry.PluginTypeDestination), org, name, version, "plugin") + localPath = registry.WithBinarySuffix(localPath) + if err := registry.DownloadPluginFromGithub(ctx, localPath, org, name, version, registry.PluginTypeDestination); err != nil { return nil, err } if err := c.newManagedClient(ctx, localPath); err != nil { return nil, err } default: - return nil, fmt.Errorf("unsupported registry %s", registry) + return nil, fmt.Errorf("unsupported registry %s", registrySpec) } return c, nil } // newManagedClient starts a new destination plugin process from local file, connects to it via gRPC server -// and returns a new DestinationClient -func (c *DestinationClient) newManagedClient(ctx context.Context, path string) error { - c.grpcSocketName = generateRandomUnixSocketName() +// and returns a new Client +func (c *Client) newManagedClient(ctx context.Context, path string) error { + c.grpcSocketName = random.GenerateRandomUnixSocketName() // spawn the plugin first and then connect args := []string{"serve", "--network", "unix", "--address", c.grpcSocketName, "--log-level", c.logger.GetLevel().String(), "--log-format", "json"} @@ -142,13 +146,13 @@ func (c *DestinationClient) newManagedClient(ctx context.Context, path string) e c.wg.Add(1) go func() { defer c.wg.Done() - lr := newLogReader(reader) + lr := logging.NewLogReader(reader) for { line, err := lr.NextLine() if errors.Is(err, io.EOF) { break } - if errors.Is(err, errLogLineToLong) { + if errors.Is(err, logging.ErrLogLineToLong) { c.logger.Err(err).Str("line", string(line)).Msg("skipping too long log line") continue } @@ -160,7 +164,7 @@ func (c *DestinationClient) newManagedClient(ctx context.Context, path string) e if err := json.Unmarshal(line, &structuredLogLine); err != nil { c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from plugin") } else { - jsonToLog(c.logger, structuredLogLine) + logging.JSONToLog(c.logger, structuredLogLine) } } }() @@ -180,8 +184,8 @@ func (c *DestinationClient) newManagedClient(ctx context.Context, path string) e return nil } -func (c *DestinationClient) GetProtocolVersion(ctx context.Context) (uint64, error) { - res, err := c.pbClient.GetProtocolVersion(ctx, &pb.GetProtocolVersion_Request{}) +func (c *Client) GetProtocolVersion(ctx context.Context) (uint64, error) { + res, err := c.pbClient.GetProtocolVersion(ctx, &pbBase.GetProtocolVersion_Request{}) if err != nil { s := status.Convert(err) if s.Code() != codes.Unimplemented { @@ -193,7 +197,7 @@ func (c *DestinationClient) GetProtocolVersion(ctx context.Context) (uint64, err return res.Version, nil } -func (c *DestinationClient) GetMetrics(ctx context.Context) (*destination.Metrics, error) { +func (c *Client) GetMetrics(ctx context.Context) (*destination.Metrics, error) { res, err := c.pbClient.GetMetrics(ctx, &pb.GetDestinationMetrics_Request{}) if err != nil { return nil, fmt.Errorf("failed to call GetMetrics: %w", err) @@ -205,28 +209,28 @@ func (c *DestinationClient) GetMetrics(ctx context.Context) (*destination.Metric return &stats, nil } -func (c *DestinationClient) Name(ctx context.Context) (string, error) { - res, err := c.pbClient.GetName(ctx, &pb.GetName_Request{}) +func (c *Client) Name(ctx context.Context) (string, error) { + res, err := c.pbClient.GetName(ctx, &pbBase.GetName_Request{}) if err != nil { return "", fmt.Errorf("failed to call GetName: %w", err) } return res.Name, nil } -func (c *DestinationClient) Version(ctx context.Context) (string, error) { - res, err := c.pbClient.GetVersion(ctx, &pb.GetVersion_Request{}) +func (c *Client) Version(ctx context.Context) (string, error) { + res, err := c.pbClient.GetVersion(ctx, &pbBase.GetVersion_Request{}) if err != nil { return "", fmt.Errorf("failed to call GetVersion: %w", err) } return res.Version, nil } -func (c *DestinationClient) Initialize(ctx context.Context, spec specs.Destination) error { +func (c *Client) Initialize(ctx context.Context, spec specs.Destination) error { b, err := json.Marshal(spec) if err != nil { return fmt.Errorf("destination configure: failed to marshal spec: %w", err) } - _, err = c.pbClient.Configure(ctx, &pb.Configure_Request{ + _, err = c.pbClient.Configure(ctx, &pbBase.Configure_Request{ Config: b, }) if err != nil { @@ -235,7 +239,7 @@ func (c *DestinationClient) Initialize(ctx context.Context, spec specs.Destinati return nil } -func (c *DestinationClient) Migrate(ctx context.Context, tables []*schema.Table) error { +func (c *Client) Migrate(ctx context.Context, tables []*schema.Table) error { b, err := json.Marshal(tables) if err != nil { return fmt.Errorf("destination migrate: failed to marshal plugin: %w", err) @@ -250,7 +254,7 @@ func (c *DestinationClient) Migrate(ctx context.Context, tables []*schema.Table) // Write writes rows as they are received from the channel to the destination plugin. // resources is marshaled schema.Resource. We are not marshalling this inside the function // because usually it is already marshalled from the destination plugin. -func (c *DestinationClient) Write(ctx context.Context, source string, syncTime time.Time, resources <-chan []byte) (uint64, error) { +func (c *Client) Write(ctx context.Context, source string, syncTime time.Time, resources <-chan []byte) (uint64, error) { saveClient, err := c.pbClient.Write(ctx) if err != nil { return 0, fmt.Errorf("failed to call Write: %w", err) @@ -276,7 +280,7 @@ func (c *DestinationClient) Write(ctx context.Context, source string, syncTime t return res.FailedWrites, nil } -func (c *DestinationClient) Write2(ctx context.Context, sourceSpec specs.Source, tables schema.Tables, syncTime time.Time, resources <-chan []byte) error { +func (c *Client) Write2(ctx context.Context, sourceSpec specs.Source, tables schema.Tables, syncTime time.Time, resources <-chan []byte) error { saveClient, err := c.pbClient.Write2(ctx) if err != nil { return fmt.Errorf("failed to call Write2: %w", err) @@ -316,14 +320,14 @@ func (c *DestinationClient) Write2(ctx context.Context, sourceSpec specs.Source, return nil } -func (c *DestinationClient) Close(ctx context.Context) error { +func (c *Client) Close(ctx context.Context) error { if _, err := c.pbClient.Close(ctx, &pb.Close_Request{}); err != nil { return fmt.Errorf("failed to close destination: %w", err) } return nil } -func (c *DestinationClient) DeleteStale(ctx context.Context, tables schema.Tables, source string, timestamp time.Time) error { +func (c *Client) DeleteStale(ctx context.Context, tables schema.Tables, source string, timestamp time.Time) error { b, err := json.Marshal(tables) if err != nil { return fmt.Errorf("destination delete stale: failed to marshal plugin: %w", err) @@ -338,9 +342,9 @@ func (c *DestinationClient) DeleteStale(ctx context.Context, tables schema.Table return nil } -// Terminate is used only in conjunction with NewManagedDestinationClient. +// Terminate is used only in conjunction with NewManagedClient. // It closes the connection it created, kills the spawned process and removes the socket file. -func (c *DestinationClient) Terminate() error { +func (c *Client) Terminate() error { // wait for log streaming to complete before returning from this function defer c.wg.Wait() diff --git a/clients/destination_terminate.go b/clients/destination/v0/destination_terminate.go similarity index 89% rename from clients/destination_terminate.go rename to clients/destination/v0/destination_terminate.go index 24bd933512..aebd7e8afc 100644 --- a/clients/destination_terminate.go +++ b/clients/destination/v0/destination_terminate.go @@ -1,6 +1,6 @@ //go:build !windows -package clients +package destination import ( "fmt" @@ -8,7 +8,7 @@ import ( "time" ) -func (c *DestinationClient) terminateProcess() error { +func (c *Client) terminateProcess() error { if err := c.cmd.Process.Signal(os.Interrupt); err != nil { c.logger.Error().Err(err).Msg("failed to send interrupt signal to destination plugin") } diff --git a/clients/destination_terminate_windows.go b/clients/destination/v0/destination_terminate_windows.go similarity index 87% rename from clients/destination_terminate_windows.go rename to clients/destination/v0/destination_terminate_windows.go index 9fa33f42d5..366d19ed24 100644 --- a/clients/destination_terminate_windows.go +++ b/clients/destination/v0/destination_terminate_windows.go @@ -1,8 +1,8 @@ //go:build windows -package clients +package destination -func (c *DestinationClient) terminateProcess() error { +func (c *Client) terminateProcess() error { if err := c.cmd.Process.Kill(); err != nil { c.logger.Error().Err(err).Msg("failed to kill destination plugin") } diff --git a/clients/destination_test.go b/clients/destination/v0/destination_test.go similarity index 91% rename from clients/destination_test.go rename to clients/destination/v0/destination_test.go index c516bfdaa9..df84afd028 100644 --- a/clients/destination_test.go +++ b/clients/destination/v0/destination_test.go @@ -1,4 +1,4 @@ -package clients +package destination import ( "context" @@ -38,7 +38,7 @@ func TestDestinationClient(t *testing.T) { for _, tc := range newDestinationClientTestCases { t.Run(tc.Path+"_"+tc.Version, func(t *testing.T) { dirName := t.TempDir() - c, err := NewDestinationClient(ctx, tc.Registry, tc.Path, tc.Version, WithDestinationLogger(l), WithDestinationDirectory(dirName)) + c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithDestinationLogger(l), WithDestinationDirectory(dirName)) if err != nil { if strings.HasPrefix(err.Error(), "destination plugin protocol version") { // this also means success as in this tests we just want to make sure we were able to download and spawn the plugin @@ -69,7 +69,7 @@ func TestDestinationClientWriteReturnsCorrectError(t *testing.T) { ctx := context.Background() l := zerolog.New(zerolog.NewTestWriter(t)).Output(zerolog.ConsoleWriter{Out: os.Stderr}).Level(zerolog.DebugLevel) dirName := t.TempDir() - c, err := NewDestinationClient(ctx, specs.RegistryGithub, "cloudquery/sqlite", "v1.0.11", WithDestinationLogger(l), WithDestinationDirectory(dirName)) + c, err := NewClient(ctx, specs.RegistryGithub, "cloudquery/sqlite", "v1.0.11", WithDestinationLogger(l), WithDestinationDirectory(dirName)) if err != nil { t.Fatal(err) } diff --git a/clients/errors.go b/clients/errors.go deleted file mode 100644 index dc12d97251..0000000000 --- a/clients/errors.go +++ /dev/null @@ -1,22 +0,0 @@ -package clients - -import ( - "errors" - - "google.golang.org/grpc/codes" - "google.golang.org/grpc/status" -) - -// IsUnimplemented returns true if an error indicates that the underlying grpc call -// was unimplemented on the server side. -func IsUnimplemented(err error) bool { - if err == nil { - return false - } - st, ok := status.FromError(err) - if ok && st.Code() == codes.Unimplemented { - return true - } - err = errors.Unwrap(err) - return IsUnimplemented(err) -} diff --git a/clients/source.go b/clients/source/v0/source.go similarity index 74% rename from clients/source.go rename to clients/source/v0/source.go index eb8632c222..14ab156549 100644 --- a/clients/source.go +++ b/clients/source/v0/source.go @@ -1,4 +1,4 @@ -package clients +package source import ( "context" @@ -13,8 +13,12 @@ import ( "strings" "sync" - "github.com/cloudquery/plugin-sdk/internal/pb" + "github.com/cloudquery/plugin-sdk/internal/logging" + pbBase "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" + pb "github.com/cloudquery/plugin-sdk/internal/pb/source/v0" + "github.com/cloudquery/plugin-sdk/internal/random" "github.com/cloudquery/plugin-sdk/plugins/source" + "github.com/cloudquery/plugin-sdk/registry" "github.com/cloudquery/plugin-sdk/schema" "github.com/cloudquery/plugin-sdk/specs" "github.com/rs/zerolog" @@ -24,8 +28,8 @@ import ( "google.golang.org/grpc/status" ) -// SourceClient -type SourceClient struct { +// Client +type Client struct { pbClient pb.SourceClient directory string cmd *exec.Cmd @@ -41,44 +45,44 @@ type FetchResultMessage struct { Resource []byte } -type SourceClientOption func(*SourceClient) +type ClientOption func(*Client) -func WithSourceLogger(logger zerolog.Logger) func(*SourceClient) { - return func(c *SourceClient) { +func WithSourceLogger(logger zerolog.Logger) func(*Client) { + return func(c *Client) { c.logger = logger } } -func WithSourceDirectory(directory string) func(*SourceClient) { - return func(c *SourceClient) { +func WithSourceDirectory(directory string) func(*Client) { + return func(c *Client) { c.directory = directory } } -func WithSourceGRPCConnection(userConn *grpc.ClientConn) func(*SourceClient) { - return func(c *SourceClient) { +func WithSourceGRPCConnection(userConn *grpc.ClientConn) func(*Client) { + return func(c *Client) { // we use a different variable here because we don't want to close a connection that wasn't created by us. c.userConn = userConn } } -func WithSourceNoSentry() func(*SourceClient) { - return func(c *SourceClient) { +func WithSourceNoSentry() func(*Client) { + return func(c *Client) { c.noSentry = true } } -// NewSourceClient connect to gRPC server running source plugin and returns a new SourceClient -func NewSourceClient(ctx context.Context, registry specs.Registry, path string, version string, opts ...SourceClientOption) (*SourceClient, error) { +// NewClient connect to gRPC server running source plugin and returns a new Client +func NewClient(ctx context.Context, registrySpec specs.Registry, path string, version string, opts ...ClientOption) (*Client, error) { var err error - c := &SourceClient{ - directory: DefaultDownloadDir, + c := &Client{ + directory: registry.DefaultDownloadDir, wg: &sync.WaitGroup{}, } for _, opt := range opts { opt(c) } - switch registry { + switch registrySpec { case specs.RegistryGrpc: if c.userConn == nil { c.conn, err = grpc.DialContext(ctx, path, grpc.WithTransportCredentials(insecure.NewCredentials())) @@ -100,25 +104,25 @@ func NewSourceClient(ctx context.Context, registry specs.Registry, path string, return nil, fmt.Errorf("invalid github plugin path: %s. format should be owner/repo", path) } org, name := pathSplit[0], pathSplit[1] - localPath := filepath.Join(c.directory, "plugins", string(PluginTypeSource), org, name, version, "plugin") - localPath = withBinarySuffix(localPath) - if err := DownloadPluginFromGithub(ctx, localPath, org, name, version, PluginTypeSource); err != nil { + localPath := filepath.Join(c.directory, "plugins", string(registry.PluginTypeSource), org, name, version, "plugin") + localPath = registry.WithBinarySuffix(localPath) + if err := registry.DownloadPluginFromGithub(ctx, localPath, org, name, version, registry.PluginTypeSource); err != nil { return nil, err } if err := c.newManagedClient(ctx, localPath); err != nil { return nil, err } default: - return nil, fmt.Errorf("unsupported registry %s", registry) + return nil, fmt.Errorf("unsupported registry %s", registrySpec) } return c, nil } // newManagedClient starts a new source plugin process from local path, connects to it via gRPC server -// and returns a new SourceClient -func (c *SourceClient) newManagedClient(ctx context.Context, path string) error { - c.grpcSocketName = generateRandomUnixSocketName() +// and returns a new Client +func (c *Client) newManagedClient(ctx context.Context, path string) error { + c.grpcSocketName = random.GenerateRandomUnixSocketName() // spawn the plugin first and then connect args := []string{"serve", "--network", "unix", "--address", c.grpcSocketName, "--log-level", c.logger.GetLevel().String(), "--log-format", "json"} @@ -139,13 +143,13 @@ func (c *SourceClient) newManagedClient(ctx context.Context, path string) error c.wg.Add(1) go func() { defer c.wg.Done() - lr := newLogReader(reader) + lr := logging.NewLogReader(reader) for { line, err := lr.NextLine() if errors.Is(err, io.EOF) { break } - if errors.Is(err, errLogLineToLong) { + if errors.Is(err, logging.ErrLogLineToLong) { c.logger.Err(err).Str("line", string(line)).Msg("skipping too long log line") continue } @@ -157,7 +161,7 @@ func (c *SourceClient) newManagedClient(ctx context.Context, path string) error if err := json.Unmarshal(line, &structuredLogLine); err != nil { c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from plugin") } else { - jsonToLog(c.logger, structuredLogLine) + logging.JSONToLog(c.logger, structuredLogLine) } } }() @@ -171,8 +175,8 @@ func (c *SourceClient) newManagedClient(ctx context.Context, path string) error grpc.WithBlock(), grpc.WithContextDialer(dialer), grpc.WithDefaultCallOptions( - grpc.MaxCallRecvMsgSize(maxMsgSize), - grpc.MaxCallSendMsgSize(maxMsgSize), + grpc.MaxCallRecvMsgSize(pb.MaxMsgSize), + grpc.MaxCallSendMsgSize(pb.MaxMsgSize), ), ) if err != nil { @@ -185,8 +189,8 @@ func (c *SourceClient) newManagedClient(ctx context.Context, path string) error return nil } -func (c *SourceClient) GetProtocolVersion(ctx context.Context) (uint64, error) { - res, err := c.pbClient.GetProtocolVersion(ctx, &pb.GetProtocolVersion_Request{}) +func (c *Client) GetProtocolVersion(ctx context.Context) (uint64, error) { + res, err := c.pbClient.GetProtocolVersion(ctx, &pbBase.GetProtocolVersion_Request{}) if err != nil { s := status.Convert(err) if s.Code() != codes.Unimplemented { @@ -198,23 +202,23 @@ func (c *SourceClient) GetProtocolVersion(ctx context.Context) (uint64, error) { return res.Version, nil } -func (c *SourceClient) Name(ctx context.Context) (string, error) { - res, err := c.pbClient.GetName(ctx, &pb.GetName_Request{}) +func (c *Client) Name(ctx context.Context) (string, error) { + res, err := c.pbClient.GetName(ctx, &pbBase.GetName_Request{}) if err != nil { return "", fmt.Errorf("failed to call GetName: %w", err) } return res.Name, nil } -func (c *SourceClient) Version(ctx context.Context) (string, error) { - res, err := c.pbClient.GetVersion(ctx, &pb.GetVersion_Request{}) +func (c *Client) Version(ctx context.Context) (string, error) { + res, err := c.pbClient.GetVersion(ctx, &pbBase.GetVersion_Request{}) if err != nil { return "", fmt.Errorf("failed to call GetVersion: %w", err) } return res.Version, nil } -func (c *SourceClient) GetMetrics(ctx context.Context) (*source.Metrics, error) { +func (c *Client) GetMetrics(ctx context.Context) (*source.Metrics, error) { res, err := c.pbClient.GetMetrics(ctx, &pb.GetSourceMetrics_Request{}) if err != nil { return nil, fmt.Errorf("failed to call GetMetrics: %w", err) @@ -226,7 +230,7 @@ func (c *SourceClient) GetMetrics(ctx context.Context) (*source.Metrics, error) return &stats, nil } -func (c *SourceClient) GetTables(ctx context.Context) ([]*schema.Table, error) { +func (c *Client) GetTables(ctx context.Context) ([]*schema.Table, error) { res, err := c.pbClient.GetTables(ctx, &pb.GetTables_Request{}) if err != nil { return nil, fmt.Errorf("failed to call GetTables: %w", err) @@ -238,7 +242,7 @@ func (c *SourceClient) GetTables(ctx context.Context) ([]*schema.Table, error) { return tables, nil } -func (c *SourceClient) GetTablesForSpec(ctx context.Context, spec *specs.Source) ([]*schema.Table, error) { +func (c *Client) GetTablesForSpec(ctx context.Context, spec *specs.Source) ([]*schema.Table, error) { var ( b []byte err error @@ -265,7 +269,7 @@ func (c *SourceClient) GetTablesForSpec(ctx context.Context, spec *specs.Source) // Sync start syncing for the source client per the given spec and returning the results // in the given channel. res is marshaled schema.Resource. We are not unmarshalling this for performance reasons // as usually this is sent over-the-wire anyway to a source plugin -func (c *SourceClient) Sync(ctx context.Context, spec specs.Source, res chan<- []byte) error { +func (c *Client) Sync(ctx context.Context, spec specs.Source, res chan<- []byte) error { b, err := json.Marshal(spec) if err != nil { return fmt.Errorf("failed to marshal source spec: %w", err) @@ -295,7 +299,7 @@ func (c *SourceClient) Sync(ctx context.Context, spec specs.Source, res chan<- [ // Sync start syncing for the source client per the given spec and returning the results // in the given channel. res is marshaled schema.Resource. We are not unmarshalling this for performance reasons // as usually this is sent over-the-wire anyway to a source plugin -func (c *SourceClient) Sync2(ctx context.Context, spec specs.Source, res chan<- []byte) error { +func (c *Client) Sync2(ctx context.Context, spec specs.Source, res chan<- []byte) error { b, err := json.Marshal(spec) if err != nil { return fmt.Errorf("failed to marshal source spec: %w", err) @@ -322,9 +326,9 @@ func (c *SourceClient) Sync2(ctx context.Context, spec specs.Source, res chan<- } } -// Terminate is used only in conjunction with NewManagedSourceClient. +// Terminate is used only in conjunction with NewManagedClient. // It closes the connection it created, kills the spawned process and removes the socket file. -func (c *SourceClient) Terminate() error { +func (c *Client) Terminate() error { // wait for log streaming to complete before returning from this function defer c.wg.Wait() @@ -351,7 +355,7 @@ func (c *SourceClient) Terminate() error { return nil } -func (c *SourceClient) GetSyncSummary(ctx context.Context) (*schema.SyncSummary, error) { +func (c *Client) GetSyncSummary(ctx context.Context) (*schema.SyncSummary, error) { res, err := c.pbClient.GetSyncSummary(ctx, &pb.GetSyncSummary_Request{}) if err != nil { return nil, fmt.Errorf("failed to call GetSyncSummary: %w", err) diff --git a/clients/source_terminate.go b/clients/source/v0/source_terminate.go similarity index 89% rename from clients/source_terminate.go rename to clients/source/v0/source_terminate.go index 5b6ed59401..11e8313fc3 100644 --- a/clients/source_terminate.go +++ b/clients/source/v0/source_terminate.go @@ -1,6 +1,6 @@ //go:build !windows -package clients +package source import ( "fmt" @@ -8,7 +8,7 @@ import ( "time" ) -func (c *SourceClient) terminateProcess() error { +func (c *Client) terminateProcess() error { if err := c.cmd.Process.Signal(os.Interrupt); err != nil { c.logger.Error().Err(err).Msg("failed to send interrupt signal to source plugin") } diff --git a/clients/source_terminate_windows.go b/clients/source/v0/source_terminate_windows.go similarity index 88% rename from clients/source_terminate_windows.go rename to clients/source/v0/source_terminate_windows.go index 99f21099b7..e5d993990b 100644 --- a/clients/source_terminate_windows.go +++ b/clients/source/v0/source_terminate_windows.go @@ -1,8 +1,8 @@ //go:build windows -package clients +package source -func (c *SourceClient) terminateProcess() error { +func (c *Client) terminateProcess() error { if err := c.cmd.Process.Kill(); err != nil { c.logger.Error().Err(err).Msg("failed to kill source plugin") } diff --git a/clients/source_test.go b/clients/source/v0/source_test.go similarity index 91% rename from clients/source_test.go rename to clients/source/v0/source_test.go index e1dea405a4..ad8985760f 100644 --- a/clients/source_test.go +++ b/clients/source/v0/source_test.go @@ -1,4 +1,4 @@ -package clients +package source import ( "context" @@ -33,7 +33,7 @@ func TestSourceClient(t *testing.T) { for _, tc := range newSourceClientTestCases { t.Run(tc.Path+"_"+tc.Version, func(t *testing.T) { dirName := t.TempDir() - c, err := NewSourceClient(ctx, tc.Registry, tc.Path, tc.Version, WithSourceLogger(l), WithSourceDirectory(dirName)) + c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithSourceLogger(l), WithSourceDirectory(dirName)) if err != nil { if strings.HasPrefix(err.Error(), "source plugin protocol version") { // this also means success as in this tests we just want to make sure we were able to download and spawn the plugin diff --git a/clients/source/v1/source.go b/clients/source/v1/source.go new file mode 100644 index 0000000000..a7dc710882 --- /dev/null +++ b/clients/source/v1/source.go @@ -0,0 +1,303 @@ +package source + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + + "github.com/cloudquery/plugin-sdk/internal/logging" + pb "github.com/cloudquery/plugin-sdk/internal/pb/source/v1" + "github.com/cloudquery/plugin-sdk/internal/random" + "github.com/cloudquery/plugin-sdk/plugins/source" + "github.com/cloudquery/plugin-sdk/registry" + "github.com/cloudquery/plugin-sdk/schema" + "github.com/cloudquery/plugin-sdk/specs" + "github.com/rs/zerolog" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" +) + +// Client +type Client struct { + pbClient pb.SourceClient + directory string + cmd *exec.Cmd + logger zerolog.Logger + userConn *grpc.ClientConn + conn *grpc.ClientConn + grpcSocketName string + noSentry bool + wg *sync.WaitGroup +} + +type FetchResultMessage struct { + Resource []byte +} + +type ClientOption func(*Client) + +func WithSourceLogger(logger zerolog.Logger) func(*Client) { + return func(c *Client) { + c.logger = logger + } +} + +func WithSourceDirectory(directory string) func(*Client) { + return func(c *Client) { + c.directory = directory + } +} + +func WithSourceGRPCConnection(userConn *grpc.ClientConn) func(*Client) { + return func(c *Client) { + // we use a different variable here because we don't want to close a connection that wasn't created by us. + c.userConn = userConn + } +} + +func WithSourceNoSentry() func(*Client) { + return func(c *Client) { + c.noSentry = true + } +} + +// NewClient connect to gRPC server running source plugin and returns a new Client +func NewClient(ctx context.Context, registrySpec specs.Registry, path string, version string, opts ...ClientOption) (*Client, error) { + var err error + c := &Client{ + directory: registry.DefaultDownloadDir, + wg: &sync.WaitGroup{}, + } + for _, opt := range opts { + opt(c) + } + switch registrySpec { + case specs.RegistryGrpc: + if c.userConn == nil { + c.conn, err = grpc.DialContext(ctx, path, grpc.WithTransportCredentials(insecure.NewCredentials())) + if err != nil { + return nil, fmt.Errorf("failed to dial grpc source plugin at %s: %w", path, err) + } + c.pbClient = pb.NewSourceClient(c.conn) + } else { + c.pbClient = pb.NewSourceClient(c.userConn) + } + return c, nil + case specs.RegistryLocal: + if err := c.newManagedClient(ctx, path); err != nil { + return nil, err + } + case specs.RegistryGithub: + pathSplit := strings.Split(path, "/") + if len(pathSplit) != 2 { + return nil, fmt.Errorf("invalid github plugin path: %s. format should be owner/repo", path) + } + org, name := pathSplit[0], pathSplit[1] + localPath := filepath.Join(c.directory, "plugins", string(registry.PluginTypeSource), org, name, version, "plugin") + localPath = registry.WithBinarySuffix(localPath) + if err := registry.DownloadPluginFromGithub(ctx, localPath, org, name, version, registry.PluginTypeSource); err != nil { + return nil, err + } + if err := c.newManagedClient(ctx, localPath); err != nil { + return nil, err + } + default: + return nil, fmt.Errorf("unsupported registry %s", registrySpec) + } + + return c, nil +} + +// newManagedClient starts a new source plugin process from local path, connects to it via gRPC server +// and returns a new Client +func (c *Client) newManagedClient(ctx context.Context, path string) error { + c.grpcSocketName = random.GenerateRandomUnixSocketName() + // spawn the plugin first and then connect + args := []string{"serve", "--network", "unix", "--address", c.grpcSocketName, + "--log-level", c.logger.GetLevel().String(), "--log-format", "json"} + if c.noSentry { + args = append(args, "--no-sentry") + } + cmd := exec.CommandContext(ctx, path, args...) + reader, err := cmd.StdoutPipe() + if err != nil { + return fmt.Errorf("failed to get stdout pipe: %w", err) + } + cmd.Stderr = os.Stderr + if err := cmd.Start(); err != nil { + return fmt.Errorf("failed to start source plugin %s: %w", path, err) + } + c.cmd = cmd + + c.wg.Add(1) + go func() { + defer c.wg.Done() + lr := logging.NewLogReader(reader) + for { + line, err := lr.NextLine() + if errors.Is(err, io.EOF) { + break + } + if errors.Is(err, logging.ErrLogLineToLong) { + c.logger.Err(err).Str("line", string(line)).Msg("skipping too long log line") + continue + } + if err != nil { + c.logger.Err(err).Msg("failed to read log line from plugin") + break + } + var structuredLogLine map[string]any + if err := json.Unmarshal(line, &structuredLogLine); err != nil { + c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from plugin") + } else { + logging.JSONToLog(c.logger, structuredLogLine) + } + } + }() + + dialer := func(ctx context.Context, addr string) (net.Conn, error) { + d := &net.Dialer{} + return d.DialContext(ctx, "unix", addr) + } + c.conn, err = grpc.DialContext(ctx, c.grpcSocketName, + grpc.WithTransportCredentials(insecure.NewCredentials()), + grpc.WithBlock(), + grpc.WithContextDialer(dialer), + grpc.WithDefaultCallOptions( + grpc.MaxCallRecvMsgSize(pb.MaxMsgSize), + grpc.MaxCallSendMsgSize(pb.MaxMsgSize), + ), + ) + if err != nil { + if err := cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill plugin process") + } + return err + } + c.pbClient = pb.NewSourceClient(c.conn) + return nil +} + +func (c *Client) Name(ctx context.Context) (string, error) { + res, err := c.pbClient.GetName(ctx, &pb.GetName_Request{}) + if err != nil { + return "", fmt.Errorf("failed to call GetName: %w", err) + } + return res.Name, nil +} + +func (c *Client) Version(ctx context.Context) (string, error) { + res, err := c.pbClient.GetVersion(ctx, &pb.GetVersion_Request{}) + if err != nil { + return "", fmt.Errorf("failed to call GetVersion: %w", err) + } + return res.Version, nil +} + +func (c *Client) GetMetrics(ctx context.Context) (*source.Metrics, error) { + res, err := c.pbClient.GetMetrics(ctx, &pb.GetMetrics_Request{}) + if err != nil { + return nil, fmt.Errorf("failed to call GetMetrics: %w", err) + } + var stats source.Metrics + if err := json.Unmarshal(res.Metrics, &stats); err != nil { + return nil, fmt.Errorf("failed to unmarshal source stats: %w", err) + } + return &stats, nil +} + +func (c *Client) GetTables(ctx context.Context) ([]*schema.Table, error) { + res, err := c.pbClient.GetTables(ctx, &pb.GetTables_Request{}) + if err != nil { + return nil, fmt.Errorf("failed to call GetTables: %w", err) + } + var tables []*schema.Table + if err := json.Unmarshal(res.Tables, &tables); err != nil { + return nil, fmt.Errorf("failed to unmarshal tables: %w", err) + } + return tables, nil +} + +func (c *Client) GetDynamicTables(ctx context.Context) ([]*schema.Table, error) { + res, err := c.pbClient.GetDynamicTables(ctx, &pb.GetDynamicTables_Request{}) + if err != nil { + return nil, fmt.Errorf("failed to call GetDynamicTables: %w", err) + } + var tables []*schema.Table + if err := json.Unmarshal(res.Tables, &tables); err != nil { + return nil, fmt.Errorf("failed to unmarshal tables: %w", err) + } + return tables, nil +} + +func (c *Client) Init(ctx context.Context, spec specs.Source) error { + b, err := json.Marshal(spec) + if err != nil { + return fmt.Errorf("failed to marshal source spec: %w", err) + } + if _, err := c.pbClient.Init(ctx, &pb.Init_Request{Spec: b}); err != nil { + return fmt.Errorf("failed to call Init: %w", err) + } + return nil +} + +// Sync start syncing for the source client per the given spec and returning the results +// in the given channel. res is marshaled schema.Resource. We are not unmarshalling this for performance reasons +// as usually this is sent over-the-wire anyway to a source plugin +func (c *Client) Sync(ctx context.Context, res chan<- []byte) error { + stream, err := c.pbClient.Sync(ctx, &pb.Sync_Request{}) + if err != nil { + return fmt.Errorf("failed to call Sync: %w", err) + } + for { + r, err := stream.Recv() + if err == io.EOF { + return nil + } + if err != nil { + return fmt.Errorf("failed to fetch resources from stream: %w", err) + } + select { + case <-ctx.Done(): + return ctx.Err() + case res <- r.Resource: + } + } +} + +// Terminate is used only in conjunction with NewManagedClient. +// It closes the connection it created, kills the spawned process and removes the socket file. +func (c *Client) Terminate() error { + // wait for log streaming to complete before returning from this function + defer c.wg.Wait() + + if c.grpcSocketName != "" { + defer func() { + if err := os.RemoveAll(c.grpcSocketName); err != nil { + c.logger.Error().Err(err).Msg("failed to remove source socket file") + } + }() + } + + if c.conn != nil { + if err := c.conn.Close(); err != nil { + c.logger.Error().Err(err).Msg("failed to close gRPC connection to source plugin") + } + c.conn = nil + } + if c.cmd != nil && c.cmd.Process != nil { + if err := c.terminateProcess(); err != nil { + return err + } + } + + return nil +} diff --git a/clients/source/v1/source_terminate.go b/clients/source/v1/source_terminate.go new file mode 100644 index 0000000000..11e8313fc3 --- /dev/null +++ b/clients/source/v1/source_terminate.go @@ -0,0 +1,30 @@ +//go:build !windows + +package source + +import ( + "fmt" + "os" + "time" +) + +func (c *Client) terminateProcess() error { + if err := c.cmd.Process.Signal(os.Interrupt); err != nil { + c.logger.Error().Err(err).Msg("failed to send interrupt signal to source plugin") + } + timer := time.AfterFunc(5*time.Second, func() { + if err := c.cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill source plugin") + } + }) + st, err := c.cmd.Process.Wait() + timer.Stop() + if err != nil { + return err + } + if !st.Success() { + return fmt.Errorf("source plugin process exited with status %s", st.String()) + } + + return nil +} diff --git a/clients/source/v1/source_terminate_windows.go b/clients/source/v1/source_terminate_windows.go new file mode 100644 index 0000000000..e5d993990b --- /dev/null +++ b/clients/source/v1/source_terminate_windows.go @@ -0,0 +1,20 @@ +//go:build windows + +package source + +func (c *Client) terminateProcess() error { + if err := c.cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill source plugin") + } + st, err := c.cmd.Process.Wait() + if err != nil { + return err + } + if !st.Success() { + // on windows there is no way to shutdown gracefully via signal. Maybe we can do it via grpc api? + // though it is a bit strange to expose api to shutdown a server :thinking?: + c.logger.Info().Msgf("source plugin process exited with %s", st.String()) + } + + return nil +} diff --git a/internal/pb/base.proto b/cloudquery/base/v0/base.proto similarity index 86% rename from internal/pb/base.proto rename to cloudquery/base/v0/base.proto index 8c4690e8e5..a75a544184 100644 --- a/internal/pb/base.proto +++ b/cloudquery/base/v0/base.proto @@ -1,6 +1,7 @@ syntax = "proto3"; package proto; -option go_package = "/pb"; + +option go_package = "github.com/cloudquery/plugin-sdk/internal/pb/base/v0;base"; message GetName { message Request {} @@ -32,4 +33,4 @@ message Configure { message Response { string error = 1; } -} \ No newline at end of file +} diff --git a/internal/pb/destination.proto b/cloudquery/destination/v0/destination.proto similarity index 93% rename from internal/pb/destination.proto rename to cloudquery/destination/v0/destination.proto index 882c7e35b2..cd803537cf 100644 --- a/internal/pb/destination.proto +++ b/cloudquery/destination/v0/destination.proto @@ -1,9 +1,11 @@ syntax = "proto3"; package proto; -option go_package = "/pb"; -import "internal/pb/base.proto"; + +import "cloudquery/base/v0/base.proto"; import "google/protobuf/timestamp.proto"; +option go_package = "github.com/cloudquery/plugin-sdk/internal/pb/destination/v0;destination"; + service Destination { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. @@ -30,15 +32,13 @@ service Destination { rpc GetMetrics(GetDestinationMetrics.Request) returns (GetDestinationMetrics.Response); } - message Migrate { message Request { string name = 1; string version = 2; bytes tables = 3; } - message Response { - } + message Response {} } message Write { @@ -64,8 +64,7 @@ message Write2 { // marshalled specs.Source bytes source_spec = 5; } - message Response { - } + message Response {} } message Close { @@ -90,4 +89,4 @@ message GetDestinationMetrics { // marshalled json of plugins.DestinationMetrics bytes metrics = 1; } -} \ No newline at end of file +} diff --git a/internal/pb/source.proto b/cloudquery/source/v0/source.proto similarity index 91% rename from internal/pb/source.proto rename to cloudquery/source/v0/source.proto index 2cef6f4bc7..40fcdb2c59 100644 --- a/internal/pb/source.proto +++ b/cloudquery/source/v0/source.proto @@ -1,13 +1,16 @@ syntax = "proto3"; package proto; -option go_package = "/pb"; -import "internal/pb/base.proto"; + +import "cloudquery/base/v0/base.proto"; import "google/protobuf/timestamp.proto"; +option go_package = "github.com/cloudquery/plugin-sdk/internal/pb/source/v0;source"; + service Source { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. // Also, on the cli side it can try to upgrade/downgrade the protocol if cli supports it. + // Depcrecated: use GetSupportedProtocolVersions instead rpc GetProtocolVersion(GetProtocolVersion.Request) returns (GetProtocolVersion.Response); // Get the name of the plugin rpc GetName(GetName.Request) returns (GetName.Response); @@ -16,6 +19,7 @@ service Source { // Get all tables the source plugin supports rpc GetTables(GetTables.Request) returns (GetTables.Response); // Get all tables the source plugin supports, optionally filtered by a given spec. + // Introduced in version 2 of the protocol. rpc GetTablesForSpec(GetTablesForSpec.Request) returns (GetTablesForSpec.Response); // GetSyncSummary returns the latest sync summary of the source plugin. we don't want to send the summary on // every sync request. @@ -87,4 +91,4 @@ message GetSourceMetrics { // marshalled json of plugins.SourceMetrics bytes metrics = 1; } -} \ No newline at end of file +} diff --git a/cloudquery/source/v1/source.proto b/cloudquery/source/v1/source.proto new file mode 100644 index 0000000000..530dc453bc --- /dev/null +++ b/cloudquery/source/v1/source.proto @@ -0,0 +1,77 @@ +syntax = "proto3"; +package cloudquery.source.v1; + +option go_package = "github.com/cloudquery/plugin-sdk/internal/pb/source/v1;source"; + +service Source { + // Get the name of the plugin + rpc GetName(GetName.Request) returns (GetName.Response); + // Get the current version of the plugin + rpc GetVersion(GetVersion.Request) returns (GetVersion.Response); + // Get all static tables the source plugin supports + rpc GetTables(GetTables.Request) returns (GetTables.Response); + // Get metrics for the source plugin + rpc GetMetrics(GetMetrics.Request) returns (GetMetrics.Response); + // Configure the plugin with the given credentials and mode + rpc Init(Init.Request) returns (Init.Response); + // Get all tables the source plugin supports. Must be called after Init + rpc GetDynamicTables(GetDynamicTables.Request) returns (GetDynamicTables.Response); + // Start the sync the source plugin + rpc Sync(Sync.Request) returns (stream Sync.Response); +} + +message GetName { + message Request {} + message Response { + string name = 1; + } +} + +message GetVersion { + message Request {} + message Response { + string version = 1; + } +} + +message Init { + message Request { + // marshaled specs.Source + bytes spec = 1; + } + message Response {} +} + +message GetDynamicTables { + message Request {} + message Response { + // marshaled []*schema.Table + bytes tables = 1; + } +} + +message Sync { + message Request {} + message Response { + // marshalled *schema.Resources + bytes resource = 1; + } +} + +message GetTables { + message Request {} + message Response { + string name = 1; + string version = 2; + // Marshalled []*schema.Table + bytes tables = 3; + } +} + +message GetMetrics { + message Request {} + message Response { + // marshalled json of plugins.SourceMetrics + bytes metrics = 1; + } +} diff --git a/clients/log_reader.go b/internal/logging/log_reader.go similarity index 85% rename from clients/log_reader.go rename to internal/logging/log_reader.go index b1b344cfb3..269f74d15e 100644 --- a/clients/log_reader.go +++ b/internal/logging/log_reader.go @@ -1,4 +1,4 @@ -package clients +package logging import ( "bufio" @@ -9,18 +9,18 @@ import ( // logReaderPrefixLen is used when returning a partial line as context in NextLine const logReaderPrefixLen = 1000 -var errLogLineToLong = errors.New("log line too long, discarding") +var ErrLogLineToLong = errors.New("log line too long, discarding") // logReader is a custom implementation similar to bufio.Scanner, but provides a way to handle lines // (or tokens) that exceed the buffer size. -type logReader struct { +type LogReader struct { bufferedReader *bufio.Reader reader io.ReadCloser // reader provided by the client } // newLogReader creates a new logReader to read log lines from an io.ReadCloser -func newLogReader(reader io.ReadCloser) *logReader { - return &logReader{ +func NewLogReader(reader io.ReadCloser) *LogReader { + return &LogReader{ reader: reader, bufferedReader: bufio.NewReader(reader), } @@ -30,7 +30,7 @@ func newLogReader(reader io.ReadCloser) *logReader { // if the end of the stream has been reached. This implementation is different from bufio.Scanner as it // also returns an error if a line is too long to fit into the buffer. In this case, an error is returned // together with a limited prefix of the line. -func (r *logReader) NextLine() ([]byte, error) { +func (r *LogReader) NextLine() ([]byte, error) { line, isPrefix, err := r.bufferedReader.ReadLine() if !isPrefix || err != nil { return line, err @@ -52,5 +52,5 @@ func (r *logReader) NextLine() ([]byte, error) { return nil, err } } - return prefix, errLogLineToLong + return prefix, ErrLogLineToLong } diff --git a/clients/log_reader_test.go b/internal/logging/log_reader_test.go similarity index 95% rename from clients/log_reader_test.go rename to internal/logging/log_reader_test.go index 674d8ea10f..94343ae22a 100644 --- a/clients/log_reader_test.go +++ b/internal/logging/log_reader_test.go @@ -1,11 +1,12 @@ -package clients +package logging import ( "bufio" - "github.com/google/go-cmp/cmp" "io" "strings" "testing" + + "github.com/google/go-cmp/cmp" ) func longStr(len int) string { @@ -52,7 +53,7 @@ func Test_LogReader(t *testing.T) { for _, tc := range cases { t.Run(tc.name, func(t *testing.T) { r := io.NopCloser(strings.NewReader(tc.text)) - lr := newLogReader(r) + lr := NewLogReader(r) var gotErr error gotLines := make([]string, 0) for i := 0; i < len(tc.wantLines)+1; i++ { @@ -98,7 +99,7 @@ func Benchmark_BufferedScanner(b *testing.B) { func Benchmark_LogReader(b *testing.B) { logs := genLogs(10, 10000) - lr := newLogReader(io.NopCloser(strings.NewReader(logs))) + lr := NewLogReader(io.NopCloser(strings.NewReader(logs))) b.ResetTimer() var got []byte for n := 0; n < b.N; n++ { diff --git a/clients/logging.go b/internal/logging/logging.go similarity index 84% rename from clients/logging.go rename to internal/logging/logging.go index 63b6ede163..c83d1cafd8 100644 --- a/clients/logging.go +++ b/internal/logging/logging.go @@ -1,8 +1,8 @@ -package clients +package logging import "github.com/rs/zerolog" -func jsonToLog(l zerolog.Logger, msg map[string]any) { +func JSONToLog(l zerolog.Logger, msg map[string]any) { level := msg["level"] delete(msg, "level") switch level { diff --git a/internal/pb/base.pb.go b/internal/pb/base/v0/base.pb.go similarity index 69% rename from internal/pb/base.pb.go rename to internal/pb/base/v0/base.pb.go index 164c4898a8..e4bb28bf63 100644 --- a/internal/pb/base.pb.go +++ b/internal/pb/base/v0/base.pb.go @@ -2,9 +2,9 @@ // versions: // protoc-gen-go v1.28.1 // protoc v3.21.12 -// source: internal/pb/base.proto +// source: cloudquery/base/v0/base.proto -package pb +package base import ( protoreflect "google.golang.org/protobuf/reflect/protoreflect" @@ -29,7 +29,7 @@ type GetName struct { func (x *GetName) Reset() { *x = GetName{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[0] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -42,7 +42,7 @@ func (x *GetName) String() string { func (*GetName) ProtoMessage() {} func (x *GetName) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[0] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -55,7 +55,7 @@ func (x *GetName) ProtoReflect() protoreflect.Message { // Deprecated: Use GetName.ProtoReflect.Descriptor instead. func (*GetName) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{0} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{0} } type GetVersion struct { @@ -67,7 +67,7 @@ type GetVersion struct { func (x *GetVersion) Reset() { *x = GetVersion{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[1] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[1] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -80,7 +80,7 @@ func (x *GetVersion) String() string { func (*GetVersion) ProtoMessage() {} func (x *GetVersion) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[1] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[1] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -93,7 +93,7 @@ func (x *GetVersion) ProtoReflect() protoreflect.Message { // Deprecated: Use GetVersion.ProtoReflect.Descriptor instead. func (*GetVersion) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{1} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{1} } type GetProtocolVersion struct { @@ -105,7 +105,7 @@ type GetProtocolVersion struct { func (x *GetProtocolVersion) Reset() { *x = GetProtocolVersion{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[2] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[2] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -118,7 +118,7 @@ func (x *GetProtocolVersion) String() string { func (*GetProtocolVersion) ProtoMessage() {} func (x *GetProtocolVersion) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[2] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[2] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -131,7 +131,7 @@ func (x *GetProtocolVersion) ProtoReflect() protoreflect.Message { // Deprecated: Use GetProtocolVersion.ProtoReflect.Descriptor instead. func (*GetProtocolVersion) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{2} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{2} } type Configure struct { @@ -143,7 +143,7 @@ type Configure struct { func (x *Configure) Reset() { *x = Configure{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[3] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[3] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -156,7 +156,7 @@ func (x *Configure) String() string { func (*Configure) ProtoMessage() {} func (x *Configure) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[3] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[3] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -169,7 +169,7 @@ func (x *Configure) ProtoReflect() protoreflect.Message { // Deprecated: Use Configure.ProtoReflect.Descriptor instead. func (*Configure) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{3} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{3} } type GetName_Request struct { @@ -181,7 +181,7 @@ type GetName_Request struct { func (x *GetName_Request) Reset() { *x = GetName_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[4] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[4] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -194,7 +194,7 @@ func (x *GetName_Request) String() string { func (*GetName_Request) ProtoMessage() {} func (x *GetName_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[4] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[4] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -207,7 +207,7 @@ func (x *GetName_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetName_Request.ProtoReflect.Descriptor instead. func (*GetName_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{0, 0} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{0, 0} } type GetName_Response struct { @@ -221,7 +221,7 @@ type GetName_Response struct { func (x *GetName_Response) Reset() { *x = GetName_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[5] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[5] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -234,7 +234,7 @@ func (x *GetName_Response) String() string { func (*GetName_Response) ProtoMessage() {} func (x *GetName_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[5] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[5] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -247,7 +247,7 @@ func (x *GetName_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetName_Response.ProtoReflect.Descriptor instead. func (*GetName_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{0, 1} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{0, 1} } func (x *GetName_Response) GetName() string { @@ -266,7 +266,7 @@ type GetVersion_Request struct { func (x *GetVersion_Request) Reset() { *x = GetVersion_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[6] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[6] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -279,7 +279,7 @@ func (x *GetVersion_Request) String() string { func (*GetVersion_Request) ProtoMessage() {} func (x *GetVersion_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[6] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[6] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -292,7 +292,7 @@ func (x *GetVersion_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetVersion_Request.ProtoReflect.Descriptor instead. func (*GetVersion_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{1, 0} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{1, 0} } type GetVersion_Response struct { @@ -306,7 +306,7 @@ type GetVersion_Response struct { func (x *GetVersion_Response) Reset() { *x = GetVersion_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[7] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[7] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -319,7 +319,7 @@ func (x *GetVersion_Response) String() string { func (*GetVersion_Response) ProtoMessage() {} func (x *GetVersion_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[7] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[7] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -332,7 +332,7 @@ func (x *GetVersion_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetVersion_Response.ProtoReflect.Descriptor instead. func (*GetVersion_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{1, 1} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{1, 1} } func (x *GetVersion_Response) GetVersion() string { @@ -351,7 +351,7 @@ type GetProtocolVersion_Request struct { func (x *GetProtocolVersion_Request) Reset() { *x = GetProtocolVersion_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[8] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[8] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -364,7 +364,7 @@ func (x *GetProtocolVersion_Request) String() string { func (*GetProtocolVersion_Request) ProtoMessage() {} func (x *GetProtocolVersion_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[8] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[8] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -377,7 +377,7 @@ func (x *GetProtocolVersion_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetProtocolVersion_Request.ProtoReflect.Descriptor instead. func (*GetProtocolVersion_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{2, 0} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{2, 0} } type GetProtocolVersion_Response struct { @@ -391,7 +391,7 @@ type GetProtocolVersion_Response struct { func (x *GetProtocolVersion_Response) Reset() { *x = GetProtocolVersion_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[9] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[9] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -404,7 +404,7 @@ func (x *GetProtocolVersion_Response) String() string { func (*GetProtocolVersion_Response) ProtoMessage() {} func (x *GetProtocolVersion_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[9] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[9] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -417,7 +417,7 @@ func (x *GetProtocolVersion_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetProtocolVersion_Response.ProtoReflect.Descriptor instead. func (*GetProtocolVersion_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{2, 1} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{2, 1} } func (x *GetProtocolVersion_Response) GetVersion() uint64 { @@ -440,7 +440,7 @@ type Configure_Request struct { func (x *Configure_Request) Reset() { *x = Configure_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[10] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[10] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -453,7 +453,7 @@ func (x *Configure_Request) String() string { func (*Configure_Request) ProtoMessage() {} func (x *Configure_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[10] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[10] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -466,7 +466,7 @@ func (x *Configure_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Configure_Request.ProtoReflect.Descriptor instead. func (*Configure_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{3, 0} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{3, 0} } func (x *Configure_Request) GetConfig() []byte { @@ -487,7 +487,7 @@ type Configure_Response struct { func (x *Configure_Response) Reset() { *x = Configure_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_base_proto_msgTypes[11] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[11] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -500,7 +500,7 @@ func (x *Configure_Response) String() string { func (*Configure_Response) ProtoMessage() {} func (x *Configure_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_base_proto_msgTypes[11] + mi := &file_cloudquery_base_v0_base_proto_msgTypes[11] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -513,7 +513,7 @@ func (x *Configure_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Configure_Response.ProtoReflect.Descriptor instead. func (*Configure_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_base_proto_rawDescGZIP(), []int{3, 1} + return file_cloudquery_base_v0_base_proto_rawDescGZIP(), []int{3, 1} } func (x *Configure_Response) GetError() string { @@ -523,45 +523,49 @@ func (x *Configure_Response) GetError() string { return "" } -var File_internal_pb_base_proto protoreflect.FileDescriptor - -var file_internal_pb_base_proto_rawDesc = []byte{ - 0x0a, 0x16, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x62, 0x61, - 0x73, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x05, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, - 0x34, 0x0a, 0x07, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, - 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, - 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3d, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, - 0x69, 0x6f, 0x6e, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, - 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, - 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, - 0x73, 0x69, 0x6f, 0x6e, 0x22, 0x45, 0x0a, 0x12, 0x47, 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, - 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, +var File_cloudquery_base_v0_base_proto protoreflect.FileDescriptor + +var file_cloudquery_base_v0_base_proto_rawDesc = []byte{ + 0x0a, 0x1d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x62, 0x61, 0x73, + 0x65, 0x2f, 0x76, 0x30, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, + 0x05, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x34, 0x0a, 0x07, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, + 0x65, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1e, 0x0a, 0x08, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, + 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x3d, 0x0a, 0x0a, + 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x01, 0x20, 0x01, - 0x28, 0x04, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x22, 0x50, 0x0a, 0x09, 0x43, - 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x65, 0x1a, 0x21, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, - 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x0c, 0x52, 0x06, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x1a, 0x20, 0x0a, 0x08, 0x52, - 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x14, 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, - 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x42, 0x05, 0x5a, - 0x03, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x22, 0x45, 0x0a, 0x12, 0x47, + 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, + 0x6e, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x0a, 0x08, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, + 0x69, 0x6f, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, + 0x6f, 0x6e, 0x22, 0x50, 0x0a, 0x09, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x65, 0x1a, + 0x21, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x63, 0x6f, + 0x6e, 0x66, 0x69, 0x67, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x63, 0x6f, 0x6e, 0x66, + 0x69, 0x67, 0x1a, 0x20, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x14, + 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x65, + 0x72, 0x72, 0x6f, 0x72, 0x42, 0x3b, 0x5a, 0x39, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, + 0x6f, 0x6d, 0x2f, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x70, 0x6c, + 0x75, 0x67, 0x69, 0x6e, 0x2d, 0x73, 0x64, 0x6b, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, + 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2f, 0x76, 0x30, 0x3b, 0x62, 0x61, 0x73, + 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( - file_internal_pb_base_proto_rawDescOnce sync.Once - file_internal_pb_base_proto_rawDescData = file_internal_pb_base_proto_rawDesc + file_cloudquery_base_v0_base_proto_rawDescOnce sync.Once + file_cloudquery_base_v0_base_proto_rawDescData = file_cloudquery_base_v0_base_proto_rawDesc ) -func file_internal_pb_base_proto_rawDescGZIP() []byte { - file_internal_pb_base_proto_rawDescOnce.Do(func() { - file_internal_pb_base_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_pb_base_proto_rawDescData) +func file_cloudquery_base_v0_base_proto_rawDescGZIP() []byte { + file_cloudquery_base_v0_base_proto_rawDescOnce.Do(func() { + file_cloudquery_base_v0_base_proto_rawDescData = protoimpl.X.CompressGZIP(file_cloudquery_base_v0_base_proto_rawDescData) }) - return file_internal_pb_base_proto_rawDescData + return file_cloudquery_base_v0_base_proto_rawDescData } -var file_internal_pb_base_proto_msgTypes = make([]protoimpl.MessageInfo, 12) -var file_internal_pb_base_proto_goTypes = []interface{}{ +var file_cloudquery_base_v0_base_proto_msgTypes = make([]protoimpl.MessageInfo, 12) +var file_cloudquery_base_v0_base_proto_goTypes = []interface{}{ (*GetName)(nil), // 0: proto.GetName (*GetVersion)(nil), // 1: proto.GetVersion (*GetProtocolVersion)(nil), // 2: proto.GetProtocolVersion @@ -575,7 +579,7 @@ var file_internal_pb_base_proto_goTypes = []interface{}{ (*Configure_Request)(nil), // 10: proto.Configure.Request (*Configure_Response)(nil), // 11: proto.Configure.Response } -var file_internal_pb_base_proto_depIdxs = []int32{ +var file_cloudquery_base_v0_base_proto_depIdxs = []int32{ 0, // [0:0] is the sub-list for method output_type 0, // [0:0] is the sub-list for method input_type 0, // [0:0] is the sub-list for extension type_name @@ -583,13 +587,13 @@ var file_internal_pb_base_proto_depIdxs = []int32{ 0, // [0:0] is the sub-list for field type_name } -func init() { file_internal_pb_base_proto_init() } -func file_internal_pb_base_proto_init() { - if File_internal_pb_base_proto != nil { +func init() { file_cloudquery_base_v0_base_proto_init() } +func file_cloudquery_base_v0_base_proto_init() { + if File_cloudquery_base_v0_base_proto != nil { return } if !protoimpl.UnsafeEnabled { - file_internal_pb_base_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetName); i { case 0: return &v.state @@ -601,7 +605,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetVersion); i { case 0: return &v.state @@ -613,7 +617,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetProtocolVersion); i { case 0: return &v.state @@ -625,7 +629,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Configure); i { case 0: return &v.state @@ -637,7 +641,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetName_Request); i { case 0: return &v.state @@ -649,7 +653,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetName_Response); i { case 0: return &v.state @@ -661,7 +665,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetVersion_Request); i { case 0: return &v.state @@ -673,7 +677,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetVersion_Response); i { case 0: return &v.state @@ -685,7 +689,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetProtocolVersion_Request); i { case 0: return &v.state @@ -697,7 +701,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetProtocolVersion_Response); i { case 0: return &v.state @@ -709,7 +713,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Configure_Request); i { case 0: return &v.state @@ -721,7 +725,7 @@ func file_internal_pb_base_proto_init() { return nil } } - file_internal_pb_base_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_base_v0_base_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Configure_Response); i { case 0: return &v.state @@ -738,18 +742,18 @@ func file_internal_pb_base_proto_init() { out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), - RawDescriptor: file_internal_pb_base_proto_rawDesc, + RawDescriptor: file_cloudquery_base_v0_base_proto_rawDesc, NumEnums: 0, NumMessages: 12, NumExtensions: 0, NumServices: 0, }, - GoTypes: file_internal_pb_base_proto_goTypes, - DependencyIndexes: file_internal_pb_base_proto_depIdxs, - MessageInfos: file_internal_pb_base_proto_msgTypes, + GoTypes: file_cloudquery_base_v0_base_proto_goTypes, + DependencyIndexes: file_cloudquery_base_v0_base_proto_depIdxs, + MessageInfos: file_cloudquery_base_v0_base_proto_msgTypes, }.Build() - File_internal_pb_base_proto = out.File - file_internal_pb_base_proto_rawDesc = nil - file_internal_pb_base_proto_goTypes = nil - file_internal_pb_base_proto_depIdxs = nil + File_cloudquery_base_v0_base_proto = out.File + file_cloudquery_base_v0_base_proto_rawDesc = nil + file_cloudquery_base_v0_base_proto_goTypes = nil + file_cloudquery_base_v0_base_proto_depIdxs = nil } diff --git a/internal/pb/destination/v0/constants.go b/internal/pb/destination/v0/constants.go new file mode 100644 index 0000000000..54d45cedfa --- /dev/null +++ b/internal/pb/destination/v0/constants.go @@ -0,0 +1,3 @@ +package destination + +const MaxMsgSize = 100 * 1024 * 1024 // 100 MiB diff --git a/internal/pb/destination.pb.go b/internal/pb/destination/v0/destination.pb.go similarity index 59% rename from internal/pb/destination.pb.go rename to internal/pb/destination/v0/destination.pb.go index 0384df4769..5002ca7286 100644 --- a/internal/pb/destination.pb.go +++ b/internal/pb/destination/v0/destination.pb.go @@ -2,11 +2,12 @@ // versions: // protoc-gen-go v1.28.1 // protoc v3.21.12 -// source: internal/pb/destination.proto +// source: cloudquery/destination/v0/destination.proto -package pb +package destination import ( + v0 "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoimpl "google.golang.org/protobuf/runtime/protoimpl" timestamppb "google.golang.org/protobuf/types/known/timestamppb" @@ -30,7 +31,7 @@ type Migrate struct { func (x *Migrate) Reset() { *x = Migrate{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[0] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -43,7 +44,7 @@ func (x *Migrate) String() string { func (*Migrate) ProtoMessage() {} func (x *Migrate) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[0] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -56,7 +57,7 @@ func (x *Migrate) ProtoReflect() protoreflect.Message { // Deprecated: Use Migrate.ProtoReflect.Descriptor instead. func (*Migrate) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{0} } type Write struct { @@ -68,7 +69,7 @@ type Write struct { func (x *Write) Reset() { *x = Write{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[1] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[1] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -81,7 +82,7 @@ func (x *Write) String() string { func (*Write) ProtoMessage() {} func (x *Write) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[1] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[1] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -94,7 +95,7 @@ func (x *Write) ProtoReflect() protoreflect.Message { // Deprecated: Use Write.ProtoReflect.Descriptor instead. func (*Write) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{1} } type Write2 struct { @@ -106,7 +107,7 @@ type Write2 struct { func (x *Write2) Reset() { *x = Write2{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[2] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[2] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -119,7 +120,7 @@ func (x *Write2) String() string { func (*Write2) ProtoMessage() {} func (x *Write2) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[2] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[2] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -132,7 +133,7 @@ func (x *Write2) ProtoReflect() protoreflect.Message { // Deprecated: Use Write2.ProtoReflect.Descriptor instead. func (*Write2) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{2} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{2} } type Close struct { @@ -144,7 +145,7 @@ type Close struct { func (x *Close) Reset() { *x = Close{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[3] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[3] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -157,7 +158,7 @@ func (x *Close) String() string { func (*Close) ProtoMessage() {} func (x *Close) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[3] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[3] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -170,7 +171,7 @@ func (x *Close) ProtoReflect() protoreflect.Message { // Deprecated: Use Close.ProtoReflect.Descriptor instead. func (*Close) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{3} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{3} } type DeleteStale struct { @@ -182,7 +183,7 @@ type DeleteStale struct { func (x *DeleteStale) Reset() { *x = DeleteStale{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[4] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[4] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -195,7 +196,7 @@ func (x *DeleteStale) String() string { func (*DeleteStale) ProtoMessage() {} func (x *DeleteStale) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[4] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[4] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -208,7 +209,7 @@ func (x *DeleteStale) ProtoReflect() protoreflect.Message { // Deprecated: Use DeleteStale.ProtoReflect.Descriptor instead. func (*DeleteStale) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{4} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{4} } type GetDestinationMetrics struct { @@ -220,7 +221,7 @@ type GetDestinationMetrics struct { func (x *GetDestinationMetrics) Reset() { *x = GetDestinationMetrics{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[5] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[5] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -233,7 +234,7 @@ func (x *GetDestinationMetrics) String() string { func (*GetDestinationMetrics) ProtoMessage() {} func (x *GetDestinationMetrics) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[5] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[5] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -246,7 +247,7 @@ func (x *GetDestinationMetrics) ProtoReflect() protoreflect.Message { // Deprecated: Use GetDestinationMetrics.ProtoReflect.Descriptor instead. func (*GetDestinationMetrics) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{5} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{5} } type Migrate_Request struct { @@ -262,7 +263,7 @@ type Migrate_Request struct { func (x *Migrate_Request) Reset() { *x = Migrate_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[6] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[6] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -275,7 +276,7 @@ func (x *Migrate_Request) String() string { func (*Migrate_Request) ProtoMessage() {} func (x *Migrate_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[6] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[6] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -288,7 +289,7 @@ func (x *Migrate_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Migrate_Request.ProtoReflect.Descriptor instead. func (*Migrate_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{0, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{0, 0} } func (x *Migrate_Request) GetName() string { @@ -321,7 +322,7 @@ type Migrate_Response struct { func (x *Migrate_Response) Reset() { *x = Migrate_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[7] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[7] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -334,7 +335,7 @@ func (x *Migrate_Response) String() string { func (*Migrate_Response) ProtoMessage() {} func (x *Migrate_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[7] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[7] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -347,7 +348,7 @@ func (x *Migrate_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Migrate_Response.ProtoReflect.Descriptor instead. func (*Migrate_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{0, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{0, 1} } type Write_Request struct { @@ -364,7 +365,7 @@ type Write_Request struct { func (x *Write_Request) Reset() { *x = Write_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[8] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[8] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -377,7 +378,7 @@ func (x *Write_Request) String() string { func (*Write_Request) ProtoMessage() {} func (x *Write_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[8] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[8] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -390,7 +391,7 @@ func (x *Write_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Write_Request.ProtoReflect.Descriptor instead. func (*Write_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{1, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{1, 0} } func (x *Write_Request) GetResource() []byte { @@ -425,7 +426,7 @@ type Write_Response struct { func (x *Write_Response) Reset() { *x = Write_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[9] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[9] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -438,7 +439,7 @@ func (x *Write_Response) String() string { func (*Write_Response) ProtoMessage() {} func (x *Write_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[9] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[9] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -451,7 +452,7 @@ func (x *Write_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Write_Response.ProtoReflect.Descriptor instead. func (*Write_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{1, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{1, 1} } func (x *Write_Response) GetFailedWrites() uint64 { @@ -479,7 +480,7 @@ type Write2_Request struct { func (x *Write2_Request) Reset() { *x = Write2_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[10] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[10] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -492,7 +493,7 @@ func (x *Write2_Request) String() string { func (*Write2_Request) ProtoMessage() {} func (x *Write2_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[10] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[10] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -505,7 +506,7 @@ func (x *Write2_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Write2_Request.ProtoReflect.Descriptor instead. func (*Write2_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{2, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{2, 0} } func (x *Write2_Request) GetSource() string { @@ -552,7 +553,7 @@ type Write2_Response struct { func (x *Write2_Response) Reset() { *x = Write2_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[11] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[11] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -565,7 +566,7 @@ func (x *Write2_Response) String() string { func (*Write2_Response) ProtoMessage() {} func (x *Write2_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[11] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[11] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -578,7 +579,7 @@ func (x *Write2_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Write2_Response.ProtoReflect.Descriptor instead. func (*Write2_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{2, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{2, 1} } type Close_Request struct { @@ -590,7 +591,7 @@ type Close_Request struct { func (x *Close_Request) Reset() { *x = Close_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[12] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[12] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -603,7 +604,7 @@ func (x *Close_Request) String() string { func (*Close_Request) ProtoMessage() {} func (x *Close_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[12] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[12] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -616,7 +617,7 @@ func (x *Close_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Close_Request.ProtoReflect.Descriptor instead. func (*Close_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{3, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{3, 0} } type Close_Response struct { @@ -628,7 +629,7 @@ type Close_Response struct { func (x *Close_Response) Reset() { *x = Close_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[13] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[13] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -641,7 +642,7 @@ func (x *Close_Response) String() string { func (*Close_Response) ProtoMessage() {} func (x *Close_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[13] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[13] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -654,7 +655,7 @@ func (x *Close_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Close_Response.ProtoReflect.Descriptor instead. func (*Close_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{3, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{3, 1} } type DeleteStale_Request struct { @@ -670,7 +671,7 @@ type DeleteStale_Request struct { func (x *DeleteStale_Request) Reset() { *x = DeleteStale_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[14] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[14] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -683,7 +684,7 @@ func (x *DeleteStale_Request) String() string { func (*DeleteStale_Request) ProtoMessage() {} func (x *DeleteStale_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[14] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[14] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -696,7 +697,7 @@ func (x *DeleteStale_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use DeleteStale_Request.ProtoReflect.Descriptor instead. func (*DeleteStale_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{4, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{4, 0} } func (x *DeleteStale_Request) GetSource() string { @@ -731,7 +732,7 @@ type DeleteStale_Response struct { func (x *DeleteStale_Response) Reset() { *x = DeleteStale_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[15] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[15] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -744,7 +745,7 @@ func (x *DeleteStale_Response) String() string { func (*DeleteStale_Response) ProtoMessage() {} func (x *DeleteStale_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[15] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[15] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -757,7 +758,7 @@ func (x *DeleteStale_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use DeleteStale_Response.ProtoReflect.Descriptor instead. func (*DeleteStale_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{4, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{4, 1} } func (x *DeleteStale_Response) GetFailedDeletes() uint64 { @@ -776,7 +777,7 @@ type GetDestinationMetrics_Request struct { func (x *GetDestinationMetrics_Request) Reset() { *x = GetDestinationMetrics_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[16] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -789,7 +790,7 @@ func (x *GetDestinationMetrics_Request) String() string { func (*GetDestinationMetrics_Request) ProtoMessage() {} func (x *GetDestinationMetrics_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[16] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[16] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -802,7 +803,7 @@ func (x *GetDestinationMetrics_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetDestinationMetrics_Request.ProtoReflect.Descriptor instead. func (*GetDestinationMetrics_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{5, 0} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{5, 0} } type GetDestinationMetrics_Response struct { @@ -817,7 +818,7 @@ type GetDestinationMetrics_Response struct { func (x *GetDestinationMetrics_Response) Reset() { *x = GetDestinationMetrics_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_destination_proto_msgTypes[17] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[17] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -830,7 +831,7 @@ func (x *GetDestinationMetrics_Response) String() string { func (*GetDestinationMetrics_Response) ProtoMessage() {} func (x *GetDestinationMetrics_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_destination_proto_msgTypes[17] + mi := &file_cloudquery_destination_v0_destination_proto_msgTypes[17] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -843,7 +844,7 @@ func (x *GetDestinationMetrics_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetDestinationMetrics_Response.ProtoReflect.Descriptor instead. func (*GetDestinationMetrics_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_destination_proto_rawDescGZIP(), []int{5, 1} + return file_cloudquery_destination_v0_destination_proto_rawDescGZIP(), []int{5, 1} } func (x *GetDestinationMetrics_Response) GetMetrics() []byte { @@ -853,124 +854,130 @@ func (x *GetDestinationMetrics_Response) GetMetrics() []byte { return nil } -var File_internal_pb_destination_proto protoreflect.FileDescriptor - -var file_internal_pb_destination_proto_rawDesc = []byte{ - 0x0a, 0x1d, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x64, 0x65, - 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, - 0x05, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x16, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, - 0x2f, 0x70, 0x62, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, - 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, - 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, - 0x66, 0x0a, 0x07, 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x1a, 0x4f, 0x0a, 0x07, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, - 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, - 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, - 0x69, 0x6f, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x18, 0x03, 0x20, - 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x1a, 0x0a, 0x0a, 0x08, 0x52, - 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0xb1, 0x01, 0x0a, 0x05, 0x57, 0x72, 0x69, 0x74, - 0x65, 0x1a, 0x77, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, - 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x08, - 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x6f, 0x75, 0x72, - 0x63, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, - 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x03, 0x20, - 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, - 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, - 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x1a, 0x2f, 0x0a, 0x08, 0x52, 0x65, - 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x23, 0x0a, 0x0d, 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, - 0x5f, 0x77, 0x72, 0x69, 0x74, 0x65, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0c, 0x66, - 0x61, 0x69, 0x6c, 0x65, 0x64, 0x57, 0x72, 0x69, 0x74, 0x65, 0x73, 0x22, 0xc7, 0x01, 0x0a, 0x06, - 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, 0x1a, 0xb0, 0x01, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, - 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x01, 0x20, 0x01, - 0x28, 0x09, 0x52, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, - 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, - 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, - 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, - 0x74, 0x61, 0x6d, 0x70, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x18, 0x03, - 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x12, 0x1a, 0x0a, 0x08, - 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x08, - 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x1f, 0x0a, 0x0b, 0x73, 0x6f, 0x75, 0x72, - 0x63, 0x65, 0x5f, 0x73, 0x70, 0x65, 0x63, 0x18, 0x05, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x0a, 0x73, - 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x70, 0x65, 0x63, 0x1a, 0x0a, 0x0a, 0x08, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x1e, 0x0a, 0x05, 0x43, 0x6c, 0x6f, 0x73, 0x65, 0x1a, 0x09, - 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x0a, 0x0a, 0x08, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0xb5, 0x01, 0x0a, 0x0b, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, - 0x53, 0x74, 0x61, 0x6c, 0x65, 0x1a, 0x73, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, - 0x12, 0x16, 0x0a, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, - 0x52, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, - 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, - 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, - 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, - 0x6d, 0x70, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, - 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x1a, 0x31, 0x0a, 0x08, 0x52, 0x65, - 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x25, 0x0a, 0x0e, 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, - 0x5f, 0x64, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x04, 0x52, 0x0d, - 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x73, 0x22, 0x48, 0x0a, - 0x15, 0x47, 0x65, 0x74, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x4d, - 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, - 0x74, 0x1a, 0x24, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, 0x0a, - 0x07, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x07, - 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x32, 0xb5, 0x05, 0x0a, 0x0b, 0x44, 0x65, 0x73, 0x74, - 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x5b, 0x0a, 0x12, 0x47, 0x65, 0x74, 0x50, 0x72, - 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x21, 0x2e, - 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, - 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, - 0x1a, 0x22, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, - 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x3a, 0x0a, 0x07, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x12, - 0x16, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x2e, - 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, - 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, - 0x12, 0x43, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x19, +var File_cloudquery_destination_v0_destination_proto protoreflect.FileDescriptor + +var file_cloudquery_destination_v0_destination_proto_rawDesc = []byte{ + 0x0a, 0x2b, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x64, 0x65, 0x73, + 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2f, 0x76, 0x30, 0x2f, 0x64, 0x65, 0x73, 0x74, + 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x05, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1d, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, + 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2f, 0x76, 0x30, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2e, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, + 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x66, 0x0a, 0x07, 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x1a, + 0x4f, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, + 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, + 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, + 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, + 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, + 0x1a, 0x0a, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0xb1, 0x01, 0x0a, + 0x05, 0x57, 0x72, 0x69, 0x74, 0x65, 0x1a, 0x77, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x0c, 0x52, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x16, 0x0a, + 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73, + 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, + 0x6d, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, + 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, + 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x1a, + 0x2f, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x23, 0x0a, 0x0d, 0x66, + 0x61, 0x69, 0x6c, 0x65, 0x64, 0x5f, 0x77, 0x72, 0x69, 0x74, 0x65, 0x73, 0x18, 0x01, 0x20, 0x01, + 0x28, 0x04, 0x52, 0x0c, 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, 0x57, 0x72, 0x69, 0x74, 0x65, 0x73, + 0x22, 0xc7, 0x01, 0x0a, 0x06, 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, 0x1a, 0xb0, 0x01, 0x0a, 0x07, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, + 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, + 0x38, 0x0a, 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x02, 0x20, 0x01, + 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, + 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, + 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, + 0x6c, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, + 0x73, 0x12, 0x1a, 0x0a, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, 0x04, 0x20, + 0x01, 0x28, 0x0c, 0x52, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x1f, 0x0a, + 0x0b, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x5f, 0x73, 0x70, 0x65, 0x63, 0x18, 0x05, 0x20, 0x01, + 0x28, 0x0c, 0x52, 0x0a, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x53, 0x70, 0x65, 0x63, 0x1a, 0x0a, + 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0x1e, 0x0a, 0x05, 0x43, 0x6c, + 0x6f, 0x73, 0x65, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x0a, + 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x22, 0xb5, 0x01, 0x0a, 0x0b, 0x44, + 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, 0x1a, 0x73, 0x0a, 0x07, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x16, 0x0a, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, + 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x06, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x12, 0x38, 0x0a, + 0x09, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, + 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, + 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x52, 0x09, 0x74, 0x69, + 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, + 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x1a, + 0x31, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x25, 0x0a, 0x0e, 0x66, + 0x61, 0x69, 0x6c, 0x65, 0x64, 0x5f, 0x64, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x73, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x04, 0x52, 0x0d, 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, 0x44, 0x65, 0x6c, 0x65, 0x74, + 0x65, 0x73, 0x22, 0x48, 0x0a, 0x15, 0x47, 0x65, 0x74, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, + 0x74, 0x69, 0x6f, 0x6e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x1a, 0x09, 0x0a, 0x07, 0x52, + 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, + 0x73, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x18, 0x01, 0x20, + 0x01, 0x28, 0x0c, 0x52, 0x07, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x32, 0xb5, 0x05, 0x0a, + 0x0b, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x12, 0x5b, 0x0a, 0x12, + 0x47, 0x65, 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, + 0x6f, 0x6e, 0x12, 0x21, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x50, 0x72, + 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x22, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, + 0x74, 0x50, 0x72, 0x6f, 0x74, 0x6f, 0x63, 0x6f, 0x6c, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, + 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x3a, 0x0a, 0x07, 0x47, 0x65, 0x74, + 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x16, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, + 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x52, 0x65, 0x73, + 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x43, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, + 0x69, 0x6f, 0x6e, 0x12, 0x19, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x56, + 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, - 0x6e, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x70, 0x72, 0x6f, 0x74, - 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, - 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x40, 0x0a, 0x09, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, - 0x72, 0x65, 0x12, 0x18, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6f, 0x6e, 0x66, 0x69, - 0x67, 0x75, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x19, 0x2e, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x65, 0x2e, 0x52, - 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x3a, 0x0a, 0x07, 0x4d, 0x69, 0x67, 0x72, 0x61, - 0x74, 0x65, 0x12, 0x16, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x4d, 0x69, 0x67, 0x72, 0x61, - 0x74, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e, 0x70, 0x72, 0x6f, - 0x74, 0x6f, 0x2e, 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, - 0x6e, 0x73, 0x65, 0x12, 0x36, 0x0a, 0x05, 0x57, 0x72, 0x69, 0x74, 0x65, 0x12, 0x14, 0x2e, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, - 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, - 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x28, 0x01, 0x12, 0x39, 0x0a, 0x06, 0x57, - 0x72, 0x69, 0x74, 0x65, 0x32, 0x12, 0x15, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, - 0x69, 0x74, 0x65, 0x32, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x70, - 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x28, 0x01, 0x12, 0x34, 0x0a, 0x05, 0x43, 0x6c, 0x6f, 0x73, 0x65, 0x12, - 0x14, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6c, 0x6f, 0x73, 0x65, 0x2e, 0x52, 0x65, - 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6c, - 0x6f, 0x73, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x46, 0x0a, 0x0b, - 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, 0x12, 0x1a, 0x2e, 0x70, 0x72, - 0x6f, 0x74, 0x6f, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, 0x2e, - 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1b, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, - 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, - 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x59, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x4d, 0x65, 0x74, 0x72, 0x69, - 0x63, 0x73, 0x12, 0x24, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x44, 0x65, - 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, - 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, - 0x2e, 0x47, 0x65, 0x74, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x4d, - 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, - 0x05, 0x5a, 0x03, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x40, 0x0a, 0x09, 0x43, 0x6f, + 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x65, 0x12, 0x18, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, + 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x75, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x19, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, + 0x75, 0x72, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x3a, 0x0a, 0x07, + 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x12, 0x16, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, + 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, + 0x17, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x4d, 0x69, 0x67, 0x72, 0x61, 0x74, 0x65, 0x2e, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x36, 0x0a, 0x05, 0x57, 0x72, 0x69, 0x74, + 0x65, 0x12, 0x14, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, 0x2e, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, + 0x57, 0x72, 0x69, 0x74, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x28, 0x01, + 0x12, 0x39, 0x0a, 0x06, 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, 0x12, 0x15, 0x2e, 0x70, 0x72, 0x6f, + 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x1a, 0x16, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x57, 0x72, 0x69, 0x74, 0x65, 0x32, + 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x28, 0x01, 0x12, 0x34, 0x0a, 0x05, 0x43, + 0x6c, 0x6f, 0x73, 0x65, 0x12, 0x14, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x43, 0x6c, 0x6f, + 0x73, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x15, 0x2e, 0x70, 0x72, 0x6f, + 0x74, 0x6f, 0x2e, 0x43, 0x6c, 0x6f, 0x73, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, + 0x65, 0x12, 0x46, 0x0a, 0x0b, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, + 0x12, 0x1a, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, + 0x74, 0x61, 0x6c, 0x65, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1b, 0x2e, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x44, 0x65, 0x6c, 0x65, 0x74, 0x65, 0x53, 0x74, 0x61, 0x6c, 0x65, + 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x59, 0x0a, 0x0a, 0x47, 0x65, 0x74, + 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x24, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, + 0x47, 0x65, 0x74, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x4d, 0x65, + 0x74, 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x25, 0x2e, + 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x44, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, + 0x74, 0x69, 0x6f, 0x6e, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, + 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x49, 0x5a, 0x47, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, + 0x6f, 0x6d, 0x2f, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x70, 0x6c, + 0x75, 0x67, 0x69, 0x6e, 0x2d, 0x73, 0x64, 0x6b, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, + 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x64, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, + 0x2f, 0x76, 0x30, 0x3b, 0x64, 0x65, 0x73, 0x74, 0x69, 0x6e, 0x61, 0x74, 0x69, 0x6f, 0x6e, 0x62, + 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( - file_internal_pb_destination_proto_rawDescOnce sync.Once - file_internal_pb_destination_proto_rawDescData = file_internal_pb_destination_proto_rawDesc + file_cloudquery_destination_v0_destination_proto_rawDescOnce sync.Once + file_cloudquery_destination_v0_destination_proto_rawDescData = file_cloudquery_destination_v0_destination_proto_rawDesc ) -func file_internal_pb_destination_proto_rawDescGZIP() []byte { - file_internal_pb_destination_proto_rawDescOnce.Do(func() { - file_internal_pb_destination_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_pb_destination_proto_rawDescData) +func file_cloudquery_destination_v0_destination_proto_rawDescGZIP() []byte { + file_cloudquery_destination_v0_destination_proto_rawDescOnce.Do(func() { + file_cloudquery_destination_v0_destination_proto_rawDescData = protoimpl.X.CompressGZIP(file_cloudquery_destination_v0_destination_proto_rawDescData) }) - return file_internal_pb_destination_proto_rawDescData + return file_cloudquery_destination_v0_destination_proto_rawDescData } -var file_internal_pb_destination_proto_msgTypes = make([]protoimpl.MessageInfo, 18) -var file_internal_pb_destination_proto_goTypes = []interface{}{ +var file_cloudquery_destination_v0_destination_proto_msgTypes = make([]protoimpl.MessageInfo, 18) +var file_cloudquery_destination_v0_destination_proto_goTypes = []interface{}{ (*Migrate)(nil), // 0: proto.Migrate (*Write)(nil), // 1: proto.Write (*Write2)(nil), // 2: proto.Write2 @@ -990,16 +997,16 @@ var file_internal_pb_destination_proto_goTypes = []interface{}{ (*GetDestinationMetrics_Request)(nil), // 16: proto.GetDestinationMetrics.Request (*GetDestinationMetrics_Response)(nil), // 17: proto.GetDestinationMetrics.Response (*timestamppb.Timestamp)(nil), // 18: google.protobuf.Timestamp - (*GetProtocolVersion_Request)(nil), // 19: proto.GetProtocolVersion.Request - (*GetName_Request)(nil), // 20: proto.GetName.Request - (*GetVersion_Request)(nil), // 21: proto.GetVersion.Request - (*Configure_Request)(nil), // 22: proto.Configure.Request - (*GetProtocolVersion_Response)(nil), // 23: proto.GetProtocolVersion.Response - (*GetName_Response)(nil), // 24: proto.GetName.Response - (*GetVersion_Response)(nil), // 25: proto.GetVersion.Response - (*Configure_Response)(nil), // 26: proto.Configure.Response -} -var file_internal_pb_destination_proto_depIdxs = []int32{ + (*v0.GetProtocolVersion_Request)(nil), // 19: proto.GetProtocolVersion.Request + (*v0.GetName_Request)(nil), // 20: proto.GetName.Request + (*v0.GetVersion_Request)(nil), // 21: proto.GetVersion.Request + (*v0.Configure_Request)(nil), // 22: proto.Configure.Request + (*v0.GetProtocolVersion_Response)(nil), // 23: proto.GetProtocolVersion.Response + (*v0.GetName_Response)(nil), // 24: proto.GetName.Response + (*v0.GetVersion_Response)(nil), // 25: proto.GetVersion.Response + (*v0.Configure_Response)(nil), // 26: proto.Configure.Response +} +var file_cloudquery_destination_v0_destination_proto_depIdxs = []int32{ 18, // 0: proto.Write.Request.timestamp:type_name -> google.protobuf.Timestamp 18, // 1: proto.Write2.Request.timestamp:type_name -> google.protobuf.Timestamp 18, // 2: proto.DeleteStale.Request.timestamp:type_name -> google.protobuf.Timestamp @@ -1030,14 +1037,13 @@ var file_internal_pb_destination_proto_depIdxs = []int32{ 0, // [0:3] is the sub-list for field type_name } -func init() { file_internal_pb_destination_proto_init() } -func file_internal_pb_destination_proto_init() { - if File_internal_pb_destination_proto != nil { +func init() { file_cloudquery_destination_v0_destination_proto_init() } +func file_cloudquery_destination_v0_destination_proto_init() { + if File_cloudquery_destination_v0_destination_proto != nil { return } - file_internal_pb_base_proto_init() if !protoimpl.UnsafeEnabled { - file_internal_pb_destination_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Migrate); i { case 0: return &v.state @@ -1049,7 +1055,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write); i { case 0: return &v.state @@ -1061,7 +1067,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write2); i { case 0: return &v.state @@ -1073,7 +1079,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Close); i { case 0: return &v.state @@ -1085,7 +1091,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*DeleteStale); i { case 0: return &v.state @@ -1097,7 +1103,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetDestinationMetrics); i { case 0: return &v.state @@ -1109,7 +1115,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Migrate_Request); i { case 0: return &v.state @@ -1121,7 +1127,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Migrate_Response); i { case 0: return &v.state @@ -1133,7 +1139,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write_Request); i { case 0: return &v.state @@ -1145,7 +1151,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write_Response); i { case 0: return &v.state @@ -1157,7 +1163,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write2_Request); i { case 0: return &v.state @@ -1169,7 +1175,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Write2_Response); i { case 0: return &v.state @@ -1181,7 +1187,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Close_Request); i { case 0: return &v.state @@ -1193,7 +1199,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Close_Response); i { case 0: return &v.state @@ -1205,7 +1211,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*DeleteStale_Request); i { case 0: return &v.state @@ -1217,7 +1223,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*DeleteStale_Response); i { case 0: return &v.state @@ -1229,7 +1235,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetDestinationMetrics_Request); i { case 0: return &v.state @@ -1241,7 +1247,7 @@ func file_internal_pb_destination_proto_init() { return nil } } - file_internal_pb_destination_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_destination_v0_destination_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetDestinationMetrics_Response); i { case 0: return &v.state @@ -1258,18 +1264,18 @@ func file_internal_pb_destination_proto_init() { out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), - RawDescriptor: file_internal_pb_destination_proto_rawDesc, + RawDescriptor: file_cloudquery_destination_v0_destination_proto_rawDesc, NumEnums: 0, NumMessages: 18, NumExtensions: 0, NumServices: 1, }, - GoTypes: file_internal_pb_destination_proto_goTypes, - DependencyIndexes: file_internal_pb_destination_proto_depIdxs, - MessageInfos: file_internal_pb_destination_proto_msgTypes, + GoTypes: file_cloudquery_destination_v0_destination_proto_goTypes, + DependencyIndexes: file_cloudquery_destination_v0_destination_proto_depIdxs, + MessageInfos: file_cloudquery_destination_v0_destination_proto_msgTypes, }.Build() - File_internal_pb_destination_proto = out.File - file_internal_pb_destination_proto_rawDesc = nil - file_internal_pb_destination_proto_goTypes = nil - file_internal_pb_destination_proto_depIdxs = nil + File_cloudquery_destination_v0_destination_proto = out.File + file_cloudquery_destination_v0_destination_proto_rawDesc = nil + file_cloudquery_destination_v0_destination_proto_goTypes = nil + file_cloudquery_destination_v0_destination_proto_depIdxs = nil } diff --git a/internal/pb/destination_grpc.pb.go b/internal/pb/destination/v0/destination_grpc.pb.go similarity index 87% rename from internal/pb/destination_grpc.pb.go rename to internal/pb/destination/v0/destination_grpc.pb.go index b77f4eb835..a8ce0d2622 100644 --- a/internal/pb/destination_grpc.pb.go +++ b/internal/pb/destination/v0/destination_grpc.pb.go @@ -2,12 +2,13 @@ // versions: // - protoc-gen-go-grpc v1.2.0 // - protoc v3.21.12 -// source: internal/pb/destination.proto +// source: cloudquery/destination/v0/destination.proto -package pb +package destination import ( context "context" + v0 "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" @@ -25,13 +26,13 @@ type DestinationClient interface { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. // Also, on the cli side it can try to upgrade/downgrade the protocol if cli supports it. - GetProtocolVersion(ctx context.Context, in *GetProtocolVersion_Request, opts ...grpc.CallOption) (*GetProtocolVersion_Response, error) + GetProtocolVersion(ctx context.Context, in *v0.GetProtocolVersion_Request, opts ...grpc.CallOption) (*v0.GetProtocolVersion_Response, error) // Get the name of the plugin - GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) + GetName(ctx context.Context, in *v0.GetName_Request, opts ...grpc.CallOption) (*v0.GetName_Response, error) // Get the current version of the plugin - GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) + GetVersion(ctx context.Context, in *v0.GetVersion_Request, opts ...grpc.CallOption) (*v0.GetVersion_Response, error) // Configure the plugin with the given credentials and mode - Configure(ctx context.Context, in *Configure_Request, opts ...grpc.CallOption) (*Configure_Response, error) + Configure(ctx context.Context, in *v0.Configure_Request, opts ...grpc.CallOption) (*v0.Configure_Response, error) // Migrate tables to the given plugin version Migrate(ctx context.Context, in *Migrate_Request, opts ...grpc.CallOption) (*Migrate_Response, error) // Write resources @@ -55,8 +56,8 @@ func NewDestinationClient(cc grpc.ClientConnInterface) DestinationClient { return &destinationClient{cc} } -func (c *destinationClient) GetProtocolVersion(ctx context.Context, in *GetProtocolVersion_Request, opts ...grpc.CallOption) (*GetProtocolVersion_Response, error) { - out := new(GetProtocolVersion_Response) +func (c *destinationClient) GetProtocolVersion(ctx context.Context, in *v0.GetProtocolVersion_Request, opts ...grpc.CallOption) (*v0.GetProtocolVersion_Response, error) { + out := new(v0.GetProtocolVersion_Response) err := c.cc.Invoke(ctx, "/proto.Destination/GetProtocolVersion", in, out, opts...) if err != nil { return nil, err @@ -64,8 +65,8 @@ func (c *destinationClient) GetProtocolVersion(ctx context.Context, in *GetProto return out, nil } -func (c *destinationClient) GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) { - out := new(GetName_Response) +func (c *destinationClient) GetName(ctx context.Context, in *v0.GetName_Request, opts ...grpc.CallOption) (*v0.GetName_Response, error) { + out := new(v0.GetName_Response) err := c.cc.Invoke(ctx, "/proto.Destination/GetName", in, out, opts...) if err != nil { return nil, err @@ -73,8 +74,8 @@ func (c *destinationClient) GetName(ctx context.Context, in *GetName_Request, op return out, nil } -func (c *destinationClient) GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) { - out := new(GetVersion_Response) +func (c *destinationClient) GetVersion(ctx context.Context, in *v0.GetVersion_Request, opts ...grpc.CallOption) (*v0.GetVersion_Response, error) { + out := new(v0.GetVersion_Response) err := c.cc.Invoke(ctx, "/proto.Destination/GetVersion", in, out, opts...) if err != nil { return nil, err @@ -82,8 +83,8 @@ func (c *destinationClient) GetVersion(ctx context.Context, in *GetVersion_Reque return out, nil } -func (c *destinationClient) Configure(ctx context.Context, in *Configure_Request, opts ...grpc.CallOption) (*Configure_Response, error) { - out := new(Configure_Response) +func (c *destinationClient) Configure(ctx context.Context, in *v0.Configure_Request, opts ...grpc.CallOption) (*v0.Configure_Response, error) { + out := new(v0.Configure_Response) err := c.cc.Invoke(ctx, "/proto.Destination/Configure", in, out, opts...) if err != nil { return nil, err @@ -202,13 +203,13 @@ type DestinationServer interface { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. // Also, on the cli side it can try to upgrade/downgrade the protocol if cli supports it. - GetProtocolVersion(context.Context, *GetProtocolVersion_Request) (*GetProtocolVersion_Response, error) + GetProtocolVersion(context.Context, *v0.GetProtocolVersion_Request) (*v0.GetProtocolVersion_Response, error) // Get the name of the plugin - GetName(context.Context, *GetName_Request) (*GetName_Response, error) + GetName(context.Context, *v0.GetName_Request) (*v0.GetName_Response, error) // Get the current version of the plugin - GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) + GetVersion(context.Context, *v0.GetVersion_Request) (*v0.GetVersion_Response, error) // Configure the plugin with the given credentials and mode - Configure(context.Context, *Configure_Request) (*Configure_Response, error) + Configure(context.Context, *v0.Configure_Request) (*v0.Configure_Response, error) // Migrate tables to the given plugin version Migrate(context.Context, *Migrate_Request) (*Migrate_Response, error) // Write resources @@ -229,16 +230,16 @@ type DestinationServer interface { type UnimplementedDestinationServer struct { } -func (UnimplementedDestinationServer) GetProtocolVersion(context.Context, *GetProtocolVersion_Request) (*GetProtocolVersion_Response, error) { +func (UnimplementedDestinationServer) GetProtocolVersion(context.Context, *v0.GetProtocolVersion_Request) (*v0.GetProtocolVersion_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetProtocolVersion not implemented") } -func (UnimplementedDestinationServer) GetName(context.Context, *GetName_Request) (*GetName_Response, error) { +func (UnimplementedDestinationServer) GetName(context.Context, *v0.GetName_Request) (*v0.GetName_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetName not implemented") } -func (UnimplementedDestinationServer) GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) { +func (UnimplementedDestinationServer) GetVersion(context.Context, *v0.GetVersion_Request) (*v0.GetVersion_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetVersion not implemented") } -func (UnimplementedDestinationServer) Configure(context.Context, *Configure_Request) (*Configure_Response, error) { +func (UnimplementedDestinationServer) Configure(context.Context, *v0.Configure_Request) (*v0.Configure_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method Configure not implemented") } func (UnimplementedDestinationServer) Migrate(context.Context, *Migrate_Request) (*Migrate_Response, error) { @@ -273,7 +274,7 @@ func RegisterDestinationServer(s grpc.ServiceRegistrar, srv DestinationServer) { } func _Destination_GetProtocolVersion_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetProtocolVersion_Request) + in := new(v0.GetProtocolVersion_Request) if err := dec(in); err != nil { return nil, err } @@ -285,13 +286,13 @@ func _Destination_GetProtocolVersion_Handler(srv interface{}, ctx context.Contex FullMethod: "/proto.Destination/GetProtocolVersion", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(DestinationServer).GetProtocolVersion(ctx, req.(*GetProtocolVersion_Request)) + return srv.(DestinationServer).GetProtocolVersion(ctx, req.(*v0.GetProtocolVersion_Request)) } return interceptor(ctx, in, info, handler) } func _Destination_GetName_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetName_Request) + in := new(v0.GetName_Request) if err := dec(in); err != nil { return nil, err } @@ -303,13 +304,13 @@ func _Destination_GetName_Handler(srv interface{}, ctx context.Context, dec func FullMethod: "/proto.Destination/GetName", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(DestinationServer).GetName(ctx, req.(*GetName_Request)) + return srv.(DestinationServer).GetName(ctx, req.(*v0.GetName_Request)) } return interceptor(ctx, in, info, handler) } func _Destination_GetVersion_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetVersion_Request) + in := new(v0.GetVersion_Request) if err := dec(in); err != nil { return nil, err } @@ -321,13 +322,13 @@ func _Destination_GetVersion_Handler(srv interface{}, ctx context.Context, dec f FullMethod: "/proto.Destination/GetVersion", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(DestinationServer).GetVersion(ctx, req.(*GetVersion_Request)) + return srv.(DestinationServer).GetVersion(ctx, req.(*v0.GetVersion_Request)) } return interceptor(ctx, in, info, handler) } func _Destination_Configure_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(Configure_Request) + in := new(v0.Configure_Request) if err := dec(in); err != nil { return nil, err } @@ -339,7 +340,7 @@ func _Destination_Configure_Handler(srv interface{}, ctx context.Context, dec fu FullMethod: "/proto.Destination/Configure", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(DestinationServer).Configure(ctx, req.(*Configure_Request)) + return srv.(DestinationServer).Configure(ctx, req.(*v0.Configure_Request)) } return interceptor(ctx, in, info, handler) } @@ -520,5 +521,5 @@ var Destination_ServiceDesc = grpc.ServiceDesc{ ClientStreams: true, }, }, - Metadata: "internal/pb/destination.proto", + Metadata: "cloudquery/destination/v0/destination.proto", } diff --git a/internal/servers/constants.go b/internal/pb/source/v0/constants.go similarity index 75% rename from internal/servers/constants.go rename to internal/pb/source/v0/constants.go index 1f74ba0207..778053472b 100644 --- a/internal/servers/constants.go +++ b/internal/pb/source/v0/constants.go @@ -1,3 +1,3 @@ -package servers +package source const MaxMsgSize = 100 * 1024 * 1024 // 100 MiB diff --git a/internal/pb/source.pb.go b/internal/pb/source/v0/source.pb.go similarity index 77% rename from internal/pb/source.pb.go rename to internal/pb/source/v0/source.pb.go index 704b2a5738..197ee80667 100644 --- a/internal/pb/source.pb.go +++ b/internal/pb/source/v0/source.pb.go @@ -2,11 +2,12 @@ // versions: // protoc-gen-go v1.28.1 // protoc v3.21.12 -// source: internal/pb/source.proto +// source: cloudquery/source/v0/source.proto -package pb +package source import ( + v0 "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoimpl "google.golang.org/protobuf/runtime/protoimpl" timestamppb "google.golang.org/protobuf/types/known/timestamppb" @@ -30,7 +31,7 @@ type Sync struct { func (x *Sync) Reset() { *x = Sync{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[0] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -43,7 +44,7 @@ func (x *Sync) String() string { func (*Sync) ProtoMessage() {} func (x *Sync) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[0] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -56,7 +57,7 @@ func (x *Sync) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync.ProtoReflect.Descriptor instead. func (*Sync) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{0} } type Sync2 struct { @@ -68,7 +69,7 @@ type Sync2 struct { func (x *Sync2) Reset() { *x = Sync2{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[1] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[1] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -81,7 +82,7 @@ func (x *Sync2) String() string { func (*Sync2) ProtoMessage() {} func (x *Sync2) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[1] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[1] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -94,7 +95,7 @@ func (x *Sync2) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync2.ProtoReflect.Descriptor instead. func (*Sync2) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{1} } type GetSyncSummary struct { @@ -106,7 +107,7 @@ type GetSyncSummary struct { func (x *GetSyncSummary) Reset() { *x = GetSyncSummary{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[2] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[2] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -119,7 +120,7 @@ func (x *GetSyncSummary) String() string { func (*GetSyncSummary) ProtoMessage() {} func (x *GetSyncSummary) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[2] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[2] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -132,7 +133,7 @@ func (x *GetSyncSummary) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSyncSummary.ProtoReflect.Descriptor instead. func (*GetSyncSummary) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{2} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{2} } type GetTables struct { @@ -144,7 +145,7 @@ type GetTables struct { func (x *GetTables) Reset() { *x = GetTables{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[3] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[3] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -157,7 +158,7 @@ func (x *GetTables) String() string { func (*GetTables) ProtoMessage() {} func (x *GetTables) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[3] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[3] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -170,7 +171,7 @@ func (x *GetTables) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTables.ProtoReflect.Descriptor instead. func (*GetTables) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{3} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{3} } type GetTablesForSpec struct { @@ -182,7 +183,7 @@ type GetTablesForSpec struct { func (x *GetTablesForSpec) Reset() { *x = GetTablesForSpec{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[4] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[4] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -195,7 +196,7 @@ func (x *GetTablesForSpec) String() string { func (*GetTablesForSpec) ProtoMessage() {} func (x *GetTablesForSpec) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[4] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[4] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -208,7 +209,7 @@ func (x *GetTablesForSpec) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTablesForSpec.ProtoReflect.Descriptor instead. func (*GetTablesForSpec) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{4} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{4} } type GetSourceMetrics struct { @@ -220,7 +221,7 @@ type GetSourceMetrics struct { func (x *GetSourceMetrics) Reset() { *x = GetSourceMetrics{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[5] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[5] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -233,7 +234,7 @@ func (x *GetSourceMetrics) String() string { func (*GetSourceMetrics) ProtoMessage() {} func (x *GetSourceMetrics) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[5] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[5] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -246,7 +247,7 @@ func (x *GetSourceMetrics) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSourceMetrics.ProtoReflect.Descriptor instead. func (*GetSourceMetrics) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{5} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{5} } type Sync_Request struct { @@ -262,7 +263,7 @@ type Sync_Request struct { func (x *Sync_Request) Reset() { *x = Sync_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[6] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[6] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -275,7 +276,7 @@ func (x *Sync_Request) String() string { func (*Sync_Request) ProtoMessage() {} func (x *Sync_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[6] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[6] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -288,7 +289,7 @@ func (x *Sync_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync_Request.ProtoReflect.Descriptor instead. func (*Sync_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{0, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{0, 0} } func (x *Sync_Request) GetSpec() []byte { @@ -317,7 +318,7 @@ type Sync_Response struct { func (x *Sync_Response) Reset() { *x = Sync_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[7] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[7] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -330,7 +331,7 @@ func (x *Sync_Response) String() string { func (*Sync_Response) ProtoMessage() {} func (x *Sync_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[7] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[7] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -343,7 +344,7 @@ func (x *Sync_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync_Response.ProtoReflect.Descriptor instead. func (*Sync_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{0, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{0, 1} } func (x *Sync_Response) GetResource() []byte { @@ -364,7 +365,7 @@ type Sync2_Request struct { func (x *Sync2_Request) Reset() { *x = Sync2_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[8] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[8] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -377,7 +378,7 @@ func (x *Sync2_Request) String() string { func (*Sync2_Request) ProtoMessage() {} func (x *Sync2_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[8] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[8] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -390,7 +391,7 @@ func (x *Sync2_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync2_Request.ProtoReflect.Descriptor instead. func (*Sync2_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{1, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{1, 0} } func (x *Sync2_Request) GetSpec() []byte { @@ -412,7 +413,7 @@ type Sync2_Response struct { func (x *Sync2_Response) Reset() { *x = Sync2_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[9] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[9] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -425,7 +426,7 @@ func (x *Sync2_Response) String() string { func (*Sync2_Response) ProtoMessage() {} func (x *Sync2_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[9] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[9] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -438,7 +439,7 @@ func (x *Sync2_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use Sync2_Response.ProtoReflect.Descriptor instead. func (*Sync2_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{1, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{1, 1} } func (x *Sync2_Response) GetResource() []byte { @@ -457,7 +458,7 @@ type GetSyncSummary_Request struct { func (x *GetSyncSummary_Request) Reset() { *x = GetSyncSummary_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[10] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[10] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -470,7 +471,7 @@ func (x *GetSyncSummary_Request) String() string { func (*GetSyncSummary_Request) ProtoMessage() {} func (x *GetSyncSummary_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[10] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[10] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -483,7 +484,7 @@ func (x *GetSyncSummary_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSyncSummary_Request.ProtoReflect.Descriptor instead. func (*GetSyncSummary_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{2, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{2, 0} } type GetSyncSummary_Response struct { @@ -498,7 +499,7 @@ type GetSyncSummary_Response struct { func (x *GetSyncSummary_Response) Reset() { *x = GetSyncSummary_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[11] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[11] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -511,7 +512,7 @@ func (x *GetSyncSummary_Response) String() string { func (*GetSyncSummary_Response) ProtoMessage() {} func (x *GetSyncSummary_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[11] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[11] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -524,7 +525,7 @@ func (x *GetSyncSummary_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSyncSummary_Response.ProtoReflect.Descriptor instead. func (*GetSyncSummary_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{2, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{2, 1} } func (x *GetSyncSummary_Response) GetSummary() []byte { @@ -543,7 +544,7 @@ type GetTables_Request struct { func (x *GetTables_Request) Reset() { *x = GetTables_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[12] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[12] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -556,7 +557,7 @@ func (x *GetTables_Request) String() string { func (*GetTables_Request) ProtoMessage() {} func (x *GetTables_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[12] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[12] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -569,7 +570,7 @@ func (x *GetTables_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTables_Request.ProtoReflect.Descriptor instead. func (*GetTables_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{3, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{3, 0} } type GetTables_Response struct { @@ -586,7 +587,7 @@ type GetTables_Response struct { func (x *GetTables_Response) Reset() { *x = GetTables_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[13] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[13] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -599,7 +600,7 @@ func (x *GetTables_Response) String() string { func (*GetTables_Response) ProtoMessage() {} func (x *GetTables_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[13] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[13] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -612,7 +613,7 @@ func (x *GetTables_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTables_Response.ProtoReflect.Descriptor instead. func (*GetTables_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{3, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{3, 1} } func (x *GetTables_Response) GetName() string { @@ -648,7 +649,7 @@ type GetTablesForSpec_Request struct { func (x *GetTablesForSpec_Request) Reset() { *x = GetTablesForSpec_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[14] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[14] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -661,7 +662,7 @@ func (x *GetTablesForSpec_Request) String() string { func (*GetTablesForSpec_Request) ProtoMessage() {} func (x *GetTablesForSpec_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[14] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[14] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -674,7 +675,7 @@ func (x *GetTablesForSpec_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTablesForSpec_Request.ProtoReflect.Descriptor instead. func (*GetTablesForSpec_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{4, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{4, 0} } func (x *GetTablesForSpec_Request) GetSpec() []byte { @@ -698,7 +699,7 @@ type GetTablesForSpec_Response struct { func (x *GetTablesForSpec_Response) Reset() { *x = GetTablesForSpec_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[15] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[15] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -711,7 +712,7 @@ func (x *GetTablesForSpec_Response) String() string { func (*GetTablesForSpec_Response) ProtoMessage() {} func (x *GetTablesForSpec_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[15] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[15] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -724,7 +725,7 @@ func (x *GetTablesForSpec_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetTablesForSpec_Response.ProtoReflect.Descriptor instead. func (*GetTablesForSpec_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{4, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{4, 1} } func (x *GetTablesForSpec_Response) GetName() string { @@ -757,7 +758,7 @@ type GetSourceMetrics_Request struct { func (x *GetSourceMetrics_Request) Reset() { *x = GetSourceMetrics_Request{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[16] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[16] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -770,7 +771,7 @@ func (x *GetSourceMetrics_Request) String() string { func (*GetSourceMetrics_Request) ProtoMessage() {} func (x *GetSourceMetrics_Request) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[16] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[16] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -783,7 +784,7 @@ func (x *GetSourceMetrics_Request) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSourceMetrics_Request.ProtoReflect.Descriptor instead. func (*GetSourceMetrics_Request) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{5, 0} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{5, 0} } type GetSourceMetrics_Response struct { @@ -798,7 +799,7 @@ type GetSourceMetrics_Response struct { func (x *GetSourceMetrics_Response) Reset() { *x = GetSourceMetrics_Response{} if protoimpl.UnsafeEnabled { - mi := &file_internal_pb_source_proto_msgTypes[17] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[17] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } @@ -811,7 +812,7 @@ func (x *GetSourceMetrics_Response) String() string { func (*GetSourceMetrics_Response) ProtoMessage() {} func (x *GetSourceMetrics_Response) ProtoReflect() protoreflect.Message { - mi := &file_internal_pb_source_proto_msgTypes[17] + mi := &file_cloudquery_source_v0_source_proto_msgTypes[17] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { @@ -824,7 +825,7 @@ func (x *GetSourceMetrics_Response) ProtoReflect() protoreflect.Message { // Deprecated: Use GetSourceMetrics_Response.ProtoReflect.Descriptor instead. func (*GetSourceMetrics_Response) Descriptor() ([]byte, []int) { - return file_internal_pb_source_proto_rawDescGZIP(), []int{5, 1} + return file_cloudquery_source_v0_source_proto_rawDescGZIP(), []int{5, 1} } func (x *GetSourceMetrics_Response) GetMetrics() []byte { @@ -834,12 +835,13 @@ func (x *GetSourceMetrics_Response) GetMetrics() []byte { return nil } -var File_internal_pb_source_proto protoreflect.FileDescriptor +var File_cloudquery_source_v0_source_proto protoreflect.FileDescriptor -var file_internal_pb_source_proto_rawDesc = []byte{ - 0x0a, 0x18, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x73, 0x6f, - 0x75, 0x72, 0x63, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x05, 0x70, 0x72, 0x6f, 0x74, - 0x6f, 0x1a, 0x16, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x62, +var file_cloudquery_source_v0_source_proto_rawDesc = []byte{ + 0x0a, 0x21, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x73, 0x6f, 0x75, + 0x72, 0x63, 0x65, 0x2f, 0x76, 0x30, 0x2f, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x12, 0x05, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1d, 0x63, 0x6c, 0x6f, 0x75, + 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2f, 0x76, 0x30, 0x2f, 0x62, 0x61, 0x73, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x87, 0x01, 0x0a, 0x04, 0x53, @@ -920,51 +922,55 @@ var file_internal_pb_source_proto_rawDesc = []byte{ 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x2e, 0x47, 0x65, 0x74, 0x53, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x4d, 0x65, 0x74, - 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x05, 0x5a, - 0x03, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, + 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x3f, 0x5a, + 0x3d, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x6c, 0x6f, 0x75, + 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2d, 0x73, 0x64, + 0x6b, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x73, 0x6f, + 0x75, 0x72, 0x63, 0x65, 0x2f, 0x76, 0x30, 0x3b, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x62, 0x06, + 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( - file_internal_pb_source_proto_rawDescOnce sync.Once - file_internal_pb_source_proto_rawDescData = file_internal_pb_source_proto_rawDesc + file_cloudquery_source_v0_source_proto_rawDescOnce sync.Once + file_cloudquery_source_v0_source_proto_rawDescData = file_cloudquery_source_v0_source_proto_rawDesc ) -func file_internal_pb_source_proto_rawDescGZIP() []byte { - file_internal_pb_source_proto_rawDescOnce.Do(func() { - file_internal_pb_source_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_pb_source_proto_rawDescData) +func file_cloudquery_source_v0_source_proto_rawDescGZIP() []byte { + file_cloudquery_source_v0_source_proto_rawDescOnce.Do(func() { + file_cloudquery_source_v0_source_proto_rawDescData = protoimpl.X.CompressGZIP(file_cloudquery_source_v0_source_proto_rawDescData) }) - return file_internal_pb_source_proto_rawDescData -} - -var file_internal_pb_source_proto_msgTypes = make([]protoimpl.MessageInfo, 18) -var file_internal_pb_source_proto_goTypes = []interface{}{ - (*Sync)(nil), // 0: proto.Sync - (*Sync2)(nil), // 1: proto.Sync2 - (*GetSyncSummary)(nil), // 2: proto.GetSyncSummary - (*GetTables)(nil), // 3: proto.GetTables - (*GetTablesForSpec)(nil), // 4: proto.GetTablesForSpec - (*GetSourceMetrics)(nil), // 5: proto.GetSourceMetrics - (*Sync_Request)(nil), // 6: proto.Sync.Request - (*Sync_Response)(nil), // 7: proto.Sync.Response - (*Sync2_Request)(nil), // 8: proto.Sync2.Request - (*Sync2_Response)(nil), // 9: proto.Sync2.Response - (*GetSyncSummary_Request)(nil), // 10: proto.GetSyncSummary.Request - (*GetSyncSummary_Response)(nil), // 11: proto.GetSyncSummary.Response - (*GetTables_Request)(nil), // 12: proto.GetTables.Request - (*GetTables_Response)(nil), // 13: proto.GetTables.Response - (*GetTablesForSpec_Request)(nil), // 14: proto.GetTablesForSpec.Request - (*GetTablesForSpec_Response)(nil), // 15: proto.GetTablesForSpec.Response - (*GetSourceMetrics_Request)(nil), // 16: proto.GetSourceMetrics.Request - (*GetSourceMetrics_Response)(nil), // 17: proto.GetSourceMetrics.Response - (*timestamppb.Timestamp)(nil), // 18: google.protobuf.Timestamp - (*GetProtocolVersion_Request)(nil), // 19: proto.GetProtocolVersion.Request - (*GetName_Request)(nil), // 20: proto.GetName.Request - (*GetVersion_Request)(nil), // 21: proto.GetVersion.Request - (*GetProtocolVersion_Response)(nil), // 22: proto.GetProtocolVersion.Response - (*GetName_Response)(nil), // 23: proto.GetName.Response - (*GetVersion_Response)(nil), // 24: proto.GetVersion.Response -} -var file_internal_pb_source_proto_depIdxs = []int32{ + return file_cloudquery_source_v0_source_proto_rawDescData +} + +var file_cloudquery_source_v0_source_proto_msgTypes = make([]protoimpl.MessageInfo, 18) +var file_cloudquery_source_v0_source_proto_goTypes = []interface{}{ + (*Sync)(nil), // 0: proto.Sync + (*Sync2)(nil), // 1: proto.Sync2 + (*GetSyncSummary)(nil), // 2: proto.GetSyncSummary + (*GetTables)(nil), // 3: proto.GetTables + (*GetTablesForSpec)(nil), // 4: proto.GetTablesForSpec + (*GetSourceMetrics)(nil), // 5: proto.GetSourceMetrics + (*Sync_Request)(nil), // 6: proto.Sync.Request + (*Sync_Response)(nil), // 7: proto.Sync.Response + (*Sync2_Request)(nil), // 8: proto.Sync2.Request + (*Sync2_Response)(nil), // 9: proto.Sync2.Response + (*GetSyncSummary_Request)(nil), // 10: proto.GetSyncSummary.Request + (*GetSyncSummary_Response)(nil), // 11: proto.GetSyncSummary.Response + (*GetTables_Request)(nil), // 12: proto.GetTables.Request + (*GetTables_Response)(nil), // 13: proto.GetTables.Response + (*GetTablesForSpec_Request)(nil), // 14: proto.GetTablesForSpec.Request + (*GetTablesForSpec_Response)(nil), // 15: proto.GetTablesForSpec.Response + (*GetSourceMetrics_Request)(nil), // 16: proto.GetSourceMetrics.Request + (*GetSourceMetrics_Response)(nil), // 17: proto.GetSourceMetrics.Response + (*timestamppb.Timestamp)(nil), // 18: google.protobuf.Timestamp + (*v0.GetProtocolVersion_Request)(nil), // 19: proto.GetProtocolVersion.Request + (*v0.GetName_Request)(nil), // 20: proto.GetName.Request + (*v0.GetVersion_Request)(nil), // 21: proto.GetVersion.Request + (*v0.GetProtocolVersion_Response)(nil), // 22: proto.GetProtocolVersion.Response + (*v0.GetName_Response)(nil), // 23: proto.GetName.Response + (*v0.GetVersion_Response)(nil), // 24: proto.GetVersion.Response +} +var file_cloudquery_source_v0_source_proto_depIdxs = []int32{ 18, // 0: proto.Sync.Request.timestamp:type_name -> google.protobuf.Timestamp 19, // 1: proto.Source.GetProtocolVersion:input_type -> proto.GetProtocolVersion.Request 20, // 2: proto.Source.GetName:input_type -> proto.GetName.Request @@ -991,14 +997,13 @@ var file_internal_pb_source_proto_depIdxs = []int32{ 0, // [0:1] is the sub-list for field type_name } -func init() { file_internal_pb_source_proto_init() } -func file_internal_pb_source_proto_init() { - if File_internal_pb_source_proto != nil { +func init() { file_cloudquery_source_v0_source_proto_init() } +func file_cloudquery_source_v0_source_proto_init() { + if File_cloudquery_source_v0_source_proto != nil { return } - file_internal_pb_base_proto_init() if !protoimpl.UnsafeEnabled { - file_internal_pb_source_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync); i { case 0: return &v.state @@ -1010,7 +1015,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync2); i { case 0: return &v.state @@ -1022,7 +1027,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSyncSummary); i { case 0: return &v.state @@ -1034,7 +1039,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTables); i { case 0: return &v.state @@ -1046,7 +1051,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTablesForSpec); i { case 0: return &v.state @@ -1058,7 +1063,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSourceMetrics); i { case 0: return &v.state @@ -1070,7 +1075,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync_Request); i { case 0: return &v.state @@ -1082,7 +1087,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync_Response); i { case 0: return &v.state @@ -1094,7 +1099,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync2_Request); i { case 0: return &v.state @@ -1106,7 +1111,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*Sync2_Response); i { case 0: return &v.state @@ -1118,7 +1123,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSyncSummary_Request); i { case 0: return &v.state @@ -1130,7 +1135,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSyncSummary_Response); i { case 0: return &v.state @@ -1142,7 +1147,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTables_Request); i { case 0: return &v.state @@ -1154,7 +1159,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTables_Response); i { case 0: return &v.state @@ -1166,7 +1171,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTablesForSpec_Request); i { case 0: return &v.state @@ -1178,7 +1183,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetTablesForSpec_Response); i { case 0: return &v.state @@ -1190,7 +1195,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSourceMetrics_Request); i { case 0: return &v.state @@ -1202,7 +1207,7 @@ func file_internal_pb_source_proto_init() { return nil } } - file_internal_pb_source_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { + file_cloudquery_source_v0_source_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*GetSourceMetrics_Response); i { case 0: return &v.state @@ -1219,18 +1224,18 @@ func file_internal_pb_source_proto_init() { out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), - RawDescriptor: file_internal_pb_source_proto_rawDesc, + RawDescriptor: file_cloudquery_source_v0_source_proto_rawDesc, NumEnums: 0, NumMessages: 18, NumExtensions: 0, NumServices: 1, }, - GoTypes: file_internal_pb_source_proto_goTypes, - DependencyIndexes: file_internal_pb_source_proto_depIdxs, - MessageInfos: file_internal_pb_source_proto_msgTypes, + GoTypes: file_cloudquery_source_v0_source_proto_goTypes, + DependencyIndexes: file_cloudquery_source_v0_source_proto_depIdxs, + MessageInfos: file_cloudquery_source_v0_source_proto_msgTypes, }.Build() - File_internal_pb_source_proto = out.File - file_internal_pb_source_proto_rawDesc = nil - file_internal_pb_source_proto_goTypes = nil - file_internal_pb_source_proto_depIdxs = nil + File_cloudquery_source_v0_source_proto = out.File + file_cloudquery_source_v0_source_proto_rawDesc = nil + file_cloudquery_source_v0_source_proto_goTypes = nil + file_cloudquery_source_v0_source_proto_depIdxs = nil } diff --git a/internal/pb/source_grpc.pb.go b/internal/pb/source/v0/source_grpc.pb.go similarity index 87% rename from internal/pb/source_grpc.pb.go rename to internal/pb/source/v0/source_grpc.pb.go index 3ae9d55226..a7d7b378de 100644 --- a/internal/pb/source_grpc.pb.go +++ b/internal/pb/source/v0/source_grpc.pb.go @@ -2,12 +2,13 @@ // versions: // - protoc-gen-go-grpc v1.2.0 // - protoc v3.21.12 -// source: internal/pb/source.proto +// source: cloudquery/source/v0/source.proto -package pb +package source import ( context "context" + v0 "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" @@ -25,14 +26,16 @@ type SourceClient interface { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. // Also, on the cli side it can try to upgrade/downgrade the protocol if cli supports it. - GetProtocolVersion(ctx context.Context, in *GetProtocolVersion_Request, opts ...grpc.CallOption) (*GetProtocolVersion_Response, error) + // Depcrecated: use GetSupportedProtocolVersions instead + GetProtocolVersion(ctx context.Context, in *v0.GetProtocolVersion_Request, opts ...grpc.CallOption) (*v0.GetProtocolVersion_Response, error) // Get the name of the plugin - GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) + GetName(ctx context.Context, in *v0.GetName_Request, opts ...grpc.CallOption) (*v0.GetName_Response, error) // Get the current version of the plugin - GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) + GetVersion(ctx context.Context, in *v0.GetVersion_Request, opts ...grpc.CallOption) (*v0.GetVersion_Response, error) // Get all tables the source plugin supports GetTables(ctx context.Context, in *GetTables_Request, opts ...grpc.CallOption) (*GetTables_Response, error) // Get all tables the source plugin supports, optionally filtered by a given spec. + // Introduced in version 2 of the protocol. GetTablesForSpec(ctx context.Context, in *GetTablesForSpec_Request, opts ...grpc.CallOption) (*GetTablesForSpec_Response, error) // GetSyncSummary returns the latest sync summary of the source plugin. we don't want to send the summary on // every sync request. @@ -53,8 +56,8 @@ func NewSourceClient(cc grpc.ClientConnInterface) SourceClient { return &sourceClient{cc} } -func (c *sourceClient) GetProtocolVersion(ctx context.Context, in *GetProtocolVersion_Request, opts ...grpc.CallOption) (*GetProtocolVersion_Response, error) { - out := new(GetProtocolVersion_Response) +func (c *sourceClient) GetProtocolVersion(ctx context.Context, in *v0.GetProtocolVersion_Request, opts ...grpc.CallOption) (*v0.GetProtocolVersion_Response, error) { + out := new(v0.GetProtocolVersion_Response) err := c.cc.Invoke(ctx, "/proto.Source/GetProtocolVersion", in, out, opts...) if err != nil { return nil, err @@ -62,8 +65,8 @@ func (c *sourceClient) GetProtocolVersion(ctx context.Context, in *GetProtocolVe return out, nil } -func (c *sourceClient) GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) { - out := new(GetName_Response) +func (c *sourceClient) GetName(ctx context.Context, in *v0.GetName_Request, opts ...grpc.CallOption) (*v0.GetName_Response, error) { + out := new(v0.GetName_Response) err := c.cc.Invoke(ctx, "/proto.Source/GetName", in, out, opts...) if err != nil { return nil, err @@ -71,8 +74,8 @@ func (c *sourceClient) GetName(ctx context.Context, in *GetName_Request, opts .. return out, nil } -func (c *sourceClient) GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) { - out := new(GetVersion_Response) +func (c *sourceClient) GetVersion(ctx context.Context, in *v0.GetVersion_Request, opts ...grpc.CallOption) (*v0.GetVersion_Response, error) { + out := new(v0.GetVersion_Response) err := c.cc.Invoke(ctx, "/proto.Source/GetVersion", in, out, opts...) if err != nil { return nil, err @@ -187,14 +190,16 @@ type SourceServer interface { // Get the current protocol version of the plugin. This helps // get the right message about upgrade/downgrade of cli and/or plugin. // Also, on the cli side it can try to upgrade/downgrade the protocol if cli supports it. - GetProtocolVersion(context.Context, *GetProtocolVersion_Request) (*GetProtocolVersion_Response, error) + // Depcrecated: use GetSupportedProtocolVersions instead + GetProtocolVersion(context.Context, *v0.GetProtocolVersion_Request) (*v0.GetProtocolVersion_Response, error) // Get the name of the plugin - GetName(context.Context, *GetName_Request) (*GetName_Response, error) + GetName(context.Context, *v0.GetName_Request) (*v0.GetName_Response, error) // Get the current version of the plugin - GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) + GetVersion(context.Context, *v0.GetVersion_Request) (*v0.GetVersion_Response, error) // Get all tables the source plugin supports GetTables(context.Context, *GetTables_Request) (*GetTables_Response, error) // Get all tables the source plugin supports, optionally filtered by a given spec. + // Introduced in version 2 of the protocol. GetTablesForSpec(context.Context, *GetTablesForSpec_Request) (*GetTablesForSpec_Response, error) // GetSyncSummary returns the latest sync summary of the source plugin. we don't want to send the summary on // every sync request. @@ -212,13 +217,13 @@ type SourceServer interface { type UnimplementedSourceServer struct { } -func (UnimplementedSourceServer) GetProtocolVersion(context.Context, *GetProtocolVersion_Request) (*GetProtocolVersion_Response, error) { +func (UnimplementedSourceServer) GetProtocolVersion(context.Context, *v0.GetProtocolVersion_Request) (*v0.GetProtocolVersion_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetProtocolVersion not implemented") } -func (UnimplementedSourceServer) GetName(context.Context, *GetName_Request) (*GetName_Response, error) { +func (UnimplementedSourceServer) GetName(context.Context, *v0.GetName_Request) (*v0.GetName_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetName not implemented") } -func (UnimplementedSourceServer) GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) { +func (UnimplementedSourceServer) GetVersion(context.Context, *v0.GetVersion_Request) (*v0.GetVersion_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetVersion not implemented") } func (UnimplementedSourceServer) GetTables(context.Context, *GetTables_Request) (*GetTables_Response, error) { @@ -253,7 +258,7 @@ func RegisterSourceServer(s grpc.ServiceRegistrar, srv SourceServer) { } func _Source_GetProtocolVersion_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetProtocolVersion_Request) + in := new(v0.GetProtocolVersion_Request) if err := dec(in); err != nil { return nil, err } @@ -265,13 +270,13 @@ func _Source_GetProtocolVersion_Handler(srv interface{}, ctx context.Context, de FullMethod: "/proto.Source/GetProtocolVersion", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(SourceServer).GetProtocolVersion(ctx, req.(*GetProtocolVersion_Request)) + return srv.(SourceServer).GetProtocolVersion(ctx, req.(*v0.GetProtocolVersion_Request)) } return interceptor(ctx, in, info, handler) } func _Source_GetName_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetName_Request) + in := new(v0.GetName_Request) if err := dec(in); err != nil { return nil, err } @@ -283,13 +288,13 @@ func _Source_GetName_Handler(srv interface{}, ctx context.Context, dec func(inte FullMethod: "/proto.Source/GetName", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(SourceServer).GetName(ctx, req.(*GetName_Request)) + return srv.(SourceServer).GetName(ctx, req.(*v0.GetName_Request)) } return interceptor(ctx, in, info, handler) } func _Source_GetVersion_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { - in := new(GetVersion_Request) + in := new(v0.GetVersion_Request) if err := dec(in); err != nil { return nil, err } @@ -301,7 +306,7 @@ func _Source_GetVersion_Handler(srv interface{}, ctx context.Context, dec func(i FullMethod: "/proto.Source/GetVersion", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { - return srv.(SourceServer).GetVersion(ctx, req.(*GetVersion_Request)) + return srv.(SourceServer).GetVersion(ctx, req.(*v0.GetVersion_Request)) } return interceptor(ctx, in, info, handler) } @@ -468,5 +473,5 @@ var Source_ServiceDesc = grpc.ServiceDesc{ ServerStreams: true, }, }, - Metadata: "internal/pb/source.proto", + Metadata: "cloudquery/source/v0/source.proto", } diff --git a/internal/pb/source/v1/constants.go b/internal/pb/source/v1/constants.go new file mode 100644 index 0000000000..778053472b --- /dev/null +++ b/internal/pb/source/v1/constants.go @@ -0,0 +1,3 @@ +package source + +const MaxMsgSize = 100 * 1024 * 1024 // 100 MiB diff --git a/internal/pb/source/v1/source.pb.go b/internal/pb/source/v1/source.pb.go new file mode 100644 index 0000000000..b61d66351e --- /dev/null +++ b/internal/pb/source/v1/source.pb.go @@ -0,0 +1,1325 @@ +// Code generated by protoc-gen-go. DO NOT EDIT. +// versions: +// protoc-gen-go v1.28.1 +// protoc v3.21.12 +// source: cloudquery/source/v1/source.proto + +package source + +import ( + protoreflect "google.golang.org/protobuf/reflect/protoreflect" + protoimpl "google.golang.org/protobuf/runtime/protoimpl" + reflect "reflect" + sync "sync" +) + +const ( + // Verify that this generated code is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) + // Verify that runtime/protoimpl is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) +) + +type GetName struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetName) Reset() { + *x = GetName{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[0] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetName) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetName) ProtoMessage() {} + +func (x *GetName) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[0] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetName.ProtoReflect.Descriptor instead. +func (*GetName) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{0} +} + +type GetVersion struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetVersion) Reset() { + *x = GetVersion{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[1] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersion) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersion) ProtoMessage() {} + +func (x *GetVersion) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[1] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersion.ProtoReflect.Descriptor instead. +func (*GetVersion) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{1} +} + +type Init struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *Init) Reset() { + *x = Init{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[2] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Init) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Init) ProtoMessage() {} + +func (x *Init) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[2] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Init.ProtoReflect.Descriptor instead. +func (*Init) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{2} +} + +type GetDynamicTables struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetDynamicTables) Reset() { + *x = GetDynamicTables{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[3] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetDynamicTables) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetDynamicTables) ProtoMessage() {} + +func (x *GetDynamicTables) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[3] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetDynamicTables.ProtoReflect.Descriptor instead. +func (*GetDynamicTables) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{3} +} + +type Sync struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *Sync) Reset() { + *x = Sync{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[4] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Sync) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Sync) ProtoMessage() {} + +func (x *Sync) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[4] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Sync.ProtoReflect.Descriptor instead. +func (*Sync) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{4} +} + +type GetTables struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetTables) Reset() { + *x = GetTables{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[5] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetTables) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetTables) ProtoMessage() {} + +func (x *GetTables) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[5] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetTables.ProtoReflect.Descriptor instead. +func (*GetTables) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{5} +} + +type GetMetrics struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetMetrics) Reset() { + *x = GetMetrics{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[6] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetMetrics) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetMetrics) ProtoMessage() {} + +func (x *GetMetrics) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[6] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetMetrics.ProtoReflect.Descriptor instead. +func (*GetMetrics) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{6} +} + +type GetName_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetName_Request) Reset() { + *x = GetName_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[7] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetName_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetName_Request) ProtoMessage() {} + +func (x *GetName_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[7] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetName_Request.ProtoReflect.Descriptor instead. +func (*GetName_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{0, 0} +} + +type GetName_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` +} + +func (x *GetName_Response) Reset() { + *x = GetName_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[8] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetName_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetName_Response) ProtoMessage() {} + +func (x *GetName_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[8] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetName_Response.ProtoReflect.Descriptor instead. +func (*GetName_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{0, 1} +} + +func (x *GetName_Response) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +type GetVersion_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetVersion_Request) Reset() { + *x = GetVersion_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[9] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersion_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersion_Request) ProtoMessage() {} + +func (x *GetVersion_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[9] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersion_Request.ProtoReflect.Descriptor instead. +func (*GetVersion_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{1, 0} +} + +type GetVersion_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Version string `protobuf:"bytes,1,opt,name=version,proto3" json:"version,omitempty"` +} + +func (x *GetVersion_Response) Reset() { + *x = GetVersion_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[10] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersion_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersion_Response) ProtoMessage() {} + +func (x *GetVersion_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[10] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersion_Response.ProtoReflect.Descriptor instead. +func (*GetVersion_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{1, 1} +} + +func (x *GetVersion_Response) GetVersion() string { + if x != nil { + return x.Version + } + return "" +} + +type Init_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // marshaled specs.Source + Spec []byte `protobuf:"bytes,1,opt,name=spec,proto3" json:"spec,omitempty"` +} + +func (x *Init_Request) Reset() { + *x = Init_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[11] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Init_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Init_Request) ProtoMessage() {} + +func (x *Init_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[11] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Init_Request.ProtoReflect.Descriptor instead. +func (*Init_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{2, 0} +} + +func (x *Init_Request) GetSpec() []byte { + if x != nil { + return x.Spec + } + return nil +} + +type Init_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *Init_Response) Reset() { + *x = Init_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[12] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Init_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Init_Response) ProtoMessage() {} + +func (x *Init_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[12] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Init_Response.ProtoReflect.Descriptor instead. +func (*Init_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{2, 1} +} + +type GetDynamicTables_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetDynamicTables_Request) Reset() { + *x = GetDynamicTables_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[13] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetDynamicTables_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetDynamicTables_Request) ProtoMessage() {} + +func (x *GetDynamicTables_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[13] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetDynamicTables_Request.ProtoReflect.Descriptor instead. +func (*GetDynamicTables_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{3, 0} +} + +type GetDynamicTables_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // marshaled []*schema.Table + Tables []byte `protobuf:"bytes,1,opt,name=tables,proto3" json:"tables,omitempty"` +} + +func (x *GetDynamicTables_Response) Reset() { + *x = GetDynamicTables_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[14] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetDynamicTables_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetDynamicTables_Response) ProtoMessage() {} + +func (x *GetDynamicTables_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[14] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetDynamicTables_Response.ProtoReflect.Descriptor instead. +func (*GetDynamicTables_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{3, 1} +} + +func (x *GetDynamicTables_Response) GetTables() []byte { + if x != nil { + return x.Tables + } + return nil +} + +type Sync_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *Sync_Request) Reset() { + *x = Sync_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[15] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Sync_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Sync_Request) ProtoMessage() {} + +func (x *Sync_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[15] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Sync_Request.ProtoReflect.Descriptor instead. +func (*Sync_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{4, 0} +} + +type Sync_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // marshalled *schema.Resources + Resource []byte `protobuf:"bytes,1,opt,name=resource,proto3" json:"resource,omitempty"` +} + +func (x *Sync_Response) Reset() { + *x = Sync_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[16] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *Sync_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*Sync_Response) ProtoMessage() {} + +func (x *Sync_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[16] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use Sync_Response.ProtoReflect.Descriptor instead. +func (*Sync_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{4, 1} +} + +func (x *Sync_Response) GetResource() []byte { + if x != nil { + return x.Resource + } + return nil +} + +type GetTables_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetTables_Request) Reset() { + *x = GetTables_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[17] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetTables_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetTables_Request) ProtoMessage() {} + +func (x *GetTables_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[17] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetTables_Request.ProtoReflect.Descriptor instead. +func (*GetTables_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{5, 0} +} + +type GetTables_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Name string `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"` + Version string `protobuf:"bytes,2,opt,name=version,proto3" json:"version,omitempty"` + // Marshalled []*schema.Table + Tables []byte `protobuf:"bytes,3,opt,name=tables,proto3" json:"tables,omitempty"` +} + +func (x *GetTables_Response) Reset() { + *x = GetTables_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[18] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetTables_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetTables_Response) ProtoMessage() {} + +func (x *GetTables_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[18] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetTables_Response.ProtoReflect.Descriptor instead. +func (*GetTables_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{5, 1} +} + +func (x *GetTables_Response) GetName() string { + if x != nil { + return x.Name + } + return "" +} + +func (x *GetTables_Response) GetVersion() string { + if x != nil { + return x.Version + } + return "" +} + +func (x *GetTables_Response) GetTables() []byte { + if x != nil { + return x.Tables + } + return nil +} + +type GetMetrics_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetMetrics_Request) Reset() { + *x = GetMetrics_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[19] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetMetrics_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetMetrics_Request) ProtoMessage() {} + +func (x *GetMetrics_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[19] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetMetrics_Request.ProtoReflect.Descriptor instead. +func (*GetMetrics_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{6, 0} +} + +type GetMetrics_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + // marshalled json of plugins.SourceMetrics + Metrics []byte `protobuf:"bytes,1,opt,name=metrics,proto3" json:"metrics,omitempty"` +} + +func (x *GetMetrics_Response) Reset() { + *x = GetMetrics_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[20] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetMetrics_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetMetrics_Response) ProtoMessage() {} + +func (x *GetMetrics_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_source_v1_source_proto_msgTypes[20] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetMetrics_Response.ProtoReflect.Descriptor instead. +func (*GetMetrics_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_source_v1_source_proto_rawDescGZIP(), []int{6, 1} +} + +func (x *GetMetrics_Response) GetMetrics() []byte { + if x != nil { + return x.Metrics + } + return nil +} + +var File_cloudquery_source_v1_source_proto protoreflect.FileDescriptor + +var file_cloudquery_source_v1_source_proto_rawDesc = []byte{ + 0x0a, 0x21, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x73, 0x6f, 0x75, + 0x72, 0x63, 0x65, 0x2f, 0x76, 0x31, 0x2f, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x70, 0x72, + 0x6f, 0x74, 0x6f, 0x12, 0x14, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, + 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x22, 0x34, 0x0a, 0x07, 0x47, 0x65, 0x74, + 0x4e, 0x61, 0x6d, 0x65, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, + 0x1e, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, + 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, + 0x3d, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x1a, 0x09, 0x0a, + 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x24, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, + 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, + 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x22, 0x31, + 0x0a, 0x04, 0x49, 0x6e, 0x69, 0x74, 0x1a, 0x1d, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, + 0x74, 0x12, 0x12, 0x0a, 0x04, 0x73, 0x70, 0x65, 0x63, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, + 0x04, 0x73, 0x70, 0x65, 0x63, 0x1a, 0x0a, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, + 0x65, 0x22, 0x41, 0x0a, 0x10, 0x47, 0x65, 0x74, 0x44, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x63, 0x54, + 0x61, 0x62, 0x6c, 0x65, 0x73, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, + 0x1a, 0x22, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x16, 0x0a, 0x06, + 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x06, 0x74, 0x61, + 0x62, 0x6c, 0x65, 0x73, 0x22, 0x39, 0x0a, 0x04, 0x53, 0x79, 0x6e, 0x63, 0x1a, 0x09, 0x0a, 0x07, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, + 0x6e, 0x73, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x18, + 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x08, 0x72, 0x65, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x22, + 0x68, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x1a, 0x09, 0x0a, 0x07, + 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x50, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, + 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, + 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x18, 0x0a, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, + 0x6f, 0x6e, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x07, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, + 0x6e, 0x12, 0x16, 0x0a, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x18, 0x03, 0x20, 0x01, 0x28, + 0x0c, 0x52, 0x06, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x22, 0x3d, 0x0a, 0x0a, 0x47, 0x65, 0x74, + 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, + 0x73, 0x74, 0x1a, 0x24, 0x0a, 0x08, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x18, + 0x0a, 0x07, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0c, 0x52, + 0x07, 0x6d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x32, 0xa1, 0x05, 0x0a, 0x06, 0x53, 0x6f, 0x75, + 0x72, 0x63, 0x65, 0x12, 0x58, 0x0a, 0x07, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x25, + 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, + 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, + 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, + 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x61, 0x0a, + 0x0a, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x12, 0x28, 0x2e, 0x63, 0x6c, + 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, + 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x29, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, + 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, + 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, + 0x12, 0x5e, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x12, 0x27, 0x2e, + 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, + 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x2e, 0x52, + 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x28, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, + 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, + 0x74, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, + 0x12, 0x61, 0x0a, 0x0a, 0x47, 0x65, 0x74, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x12, 0x28, + 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, + 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x47, 0x65, 0x74, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, + 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x29, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, + 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, + 0x47, 0x65, 0x74, 0x4d, 0x65, 0x74, 0x72, 0x69, 0x63, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, + 0x6e, 0x73, 0x65, 0x12, 0x4f, 0x0a, 0x04, 0x49, 0x6e, 0x69, 0x74, 0x12, 0x22, 0x2e, 0x63, 0x6c, + 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, + 0x76, 0x31, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, + 0x23, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, + 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x2e, 0x52, 0x65, 0x73, 0x70, + 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x73, 0x0a, 0x10, 0x47, 0x65, 0x74, 0x44, 0x79, 0x6e, 0x61, 0x6d, + 0x69, 0x63, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, 0x12, 0x2e, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, + 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, + 0x47, 0x65, 0x74, 0x44, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x63, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, + 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2f, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, + 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, + 0x47, 0x65, 0x74, 0x44, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x63, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x73, + 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x51, 0x0a, 0x04, 0x53, 0x79, 0x6e, + 0x63, 0x12, 0x22, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x73, + 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x53, 0x79, 0x6e, 0x63, 0x2e, 0x52, 0x65, + 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x23, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, + 0x72, 0x79, 0x2e, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x2e, 0x76, 0x31, 0x2e, 0x53, 0x79, 0x6e, + 0x63, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x30, 0x01, 0x42, 0x3f, 0x5a, 0x3d, + 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x6c, 0x6f, 0x75, 0x64, + 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2d, 0x73, 0x64, 0x6b, + 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x73, 0x6f, 0x75, + 0x72, 0x63, 0x65, 0x2f, 0x76, 0x31, 0x3b, 0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0x62, 0x06, 0x70, + 0x72, 0x6f, 0x74, 0x6f, 0x33, +} + +var ( + file_cloudquery_source_v1_source_proto_rawDescOnce sync.Once + file_cloudquery_source_v1_source_proto_rawDescData = file_cloudquery_source_v1_source_proto_rawDesc +) + +func file_cloudquery_source_v1_source_proto_rawDescGZIP() []byte { + file_cloudquery_source_v1_source_proto_rawDescOnce.Do(func() { + file_cloudquery_source_v1_source_proto_rawDescData = protoimpl.X.CompressGZIP(file_cloudquery_source_v1_source_proto_rawDescData) + }) + return file_cloudquery_source_v1_source_proto_rawDescData +} + +var file_cloudquery_source_v1_source_proto_msgTypes = make([]protoimpl.MessageInfo, 21) +var file_cloudquery_source_v1_source_proto_goTypes = []interface{}{ + (*GetName)(nil), // 0: cloudquery.source.v1.GetName + (*GetVersion)(nil), // 1: cloudquery.source.v1.GetVersion + (*Init)(nil), // 2: cloudquery.source.v1.Init + (*GetDynamicTables)(nil), // 3: cloudquery.source.v1.GetDynamicTables + (*Sync)(nil), // 4: cloudquery.source.v1.Sync + (*GetTables)(nil), // 5: cloudquery.source.v1.GetTables + (*GetMetrics)(nil), // 6: cloudquery.source.v1.GetMetrics + (*GetName_Request)(nil), // 7: cloudquery.source.v1.GetName.Request + (*GetName_Response)(nil), // 8: cloudquery.source.v1.GetName.Response + (*GetVersion_Request)(nil), // 9: cloudquery.source.v1.GetVersion.Request + (*GetVersion_Response)(nil), // 10: cloudquery.source.v1.GetVersion.Response + (*Init_Request)(nil), // 11: cloudquery.source.v1.Init.Request + (*Init_Response)(nil), // 12: cloudquery.source.v1.Init.Response + (*GetDynamicTables_Request)(nil), // 13: cloudquery.source.v1.GetDynamicTables.Request + (*GetDynamicTables_Response)(nil), // 14: cloudquery.source.v1.GetDynamicTables.Response + (*Sync_Request)(nil), // 15: cloudquery.source.v1.Sync.Request + (*Sync_Response)(nil), // 16: cloudquery.source.v1.Sync.Response + (*GetTables_Request)(nil), // 17: cloudquery.source.v1.GetTables.Request + (*GetTables_Response)(nil), // 18: cloudquery.source.v1.GetTables.Response + (*GetMetrics_Request)(nil), // 19: cloudquery.source.v1.GetMetrics.Request + (*GetMetrics_Response)(nil), // 20: cloudquery.source.v1.GetMetrics.Response +} +var file_cloudquery_source_v1_source_proto_depIdxs = []int32{ + 7, // 0: cloudquery.source.v1.Source.GetName:input_type -> cloudquery.source.v1.GetName.Request + 9, // 1: cloudquery.source.v1.Source.GetVersion:input_type -> cloudquery.source.v1.GetVersion.Request + 17, // 2: cloudquery.source.v1.Source.GetTables:input_type -> cloudquery.source.v1.GetTables.Request + 19, // 3: cloudquery.source.v1.Source.GetMetrics:input_type -> cloudquery.source.v1.GetMetrics.Request + 11, // 4: cloudquery.source.v1.Source.Init:input_type -> cloudquery.source.v1.Init.Request + 13, // 5: cloudquery.source.v1.Source.GetDynamicTables:input_type -> cloudquery.source.v1.GetDynamicTables.Request + 15, // 6: cloudquery.source.v1.Source.Sync:input_type -> cloudquery.source.v1.Sync.Request + 8, // 7: cloudquery.source.v1.Source.GetName:output_type -> cloudquery.source.v1.GetName.Response + 10, // 8: cloudquery.source.v1.Source.GetVersion:output_type -> cloudquery.source.v1.GetVersion.Response + 18, // 9: cloudquery.source.v1.Source.GetTables:output_type -> cloudquery.source.v1.GetTables.Response + 20, // 10: cloudquery.source.v1.Source.GetMetrics:output_type -> cloudquery.source.v1.GetMetrics.Response + 12, // 11: cloudquery.source.v1.Source.Init:output_type -> cloudquery.source.v1.Init.Response + 14, // 12: cloudquery.source.v1.Source.GetDynamicTables:output_type -> cloudquery.source.v1.GetDynamicTables.Response + 16, // 13: cloudquery.source.v1.Source.Sync:output_type -> cloudquery.source.v1.Sync.Response + 7, // [7:14] is the sub-list for method output_type + 0, // [0:7] is the sub-list for method input_type + 0, // [0:0] is the sub-list for extension type_name + 0, // [0:0] is the sub-list for extension extendee + 0, // [0:0] is the sub-list for field type_name +} + +func init() { file_cloudquery_source_v1_source_proto_init() } +func file_cloudquery_source_v1_source_proto_init() { + if File_cloudquery_source_v1_source_proto != nil { + return + } + if !protoimpl.UnsafeEnabled { + file_cloudquery_source_v1_source_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetName); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersion); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Init); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[3].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetDynamicTables); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[4].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Sync); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[5].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetTables); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[6].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetMetrics); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[7].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetName_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[8].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetName_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[9].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersion_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[10].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersion_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[11].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Init_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[12].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Init_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[13].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetDynamicTables_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[14].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetDynamicTables_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[15].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Sync_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[16].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*Sync_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[17].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetTables_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[18].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetTables_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[19].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetMetrics_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_source_v1_source_proto_msgTypes[20].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetMetrics_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + } + type x struct{} + out := protoimpl.TypeBuilder{ + File: protoimpl.DescBuilder{ + GoPackagePath: reflect.TypeOf(x{}).PkgPath(), + RawDescriptor: file_cloudquery_source_v1_source_proto_rawDesc, + NumEnums: 0, + NumMessages: 21, + NumExtensions: 0, + NumServices: 1, + }, + GoTypes: file_cloudquery_source_v1_source_proto_goTypes, + DependencyIndexes: file_cloudquery_source_v1_source_proto_depIdxs, + MessageInfos: file_cloudquery_source_v1_source_proto_msgTypes, + }.Build() + File_cloudquery_source_v1_source_proto = out.File + file_cloudquery_source_v1_source_proto_rawDesc = nil + file_cloudquery_source_v1_source_proto_goTypes = nil + file_cloudquery_source_v1_source_proto_depIdxs = nil +} diff --git a/internal/pb/source/v1/source_grpc.pb.go b/internal/pb/source/v1/source_grpc.pb.go new file mode 100644 index 0000000000..d1cfd2592a --- /dev/null +++ b/internal/pb/source/v1/source_grpc.pb.go @@ -0,0 +1,363 @@ +// Code generated by protoc-gen-go-grpc. DO NOT EDIT. +// versions: +// - protoc-gen-go-grpc v1.2.0 +// - protoc v3.21.12 +// source: cloudquery/source/v1/source.proto + +package source + +import ( + context "context" + grpc "google.golang.org/grpc" + codes "google.golang.org/grpc/codes" + status "google.golang.org/grpc/status" +) + +// This is a compile-time assertion to ensure that this generated file +// is compatible with the grpc package it is being compiled against. +// Requires gRPC-Go v1.32.0 or later. +const _ = grpc.SupportPackageIsVersion7 + +// SourceClient is the client API for Source service. +// +// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. +type SourceClient interface { + // Get the name of the plugin + GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) + // Get the current version of the plugin + GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) + // Get all static tables the source plugin supports + GetTables(ctx context.Context, in *GetTables_Request, opts ...grpc.CallOption) (*GetTables_Response, error) + // Get metrics for the source plugin + GetMetrics(ctx context.Context, in *GetMetrics_Request, opts ...grpc.CallOption) (*GetMetrics_Response, error) + // Configure the plugin with the given credentials and mode + Init(ctx context.Context, in *Init_Request, opts ...grpc.CallOption) (*Init_Response, error) + // Get all tables the source plugin supports. Must be called after Init + GetDynamicTables(ctx context.Context, in *GetDynamicTables_Request, opts ...grpc.CallOption) (*GetDynamicTables_Response, error) + // Start the sync the source plugin + Sync(ctx context.Context, in *Sync_Request, opts ...grpc.CallOption) (Source_SyncClient, error) +} + +type sourceClient struct { + cc grpc.ClientConnInterface +} + +func NewSourceClient(cc grpc.ClientConnInterface) SourceClient { + return &sourceClient{cc} +} + +func (c *sourceClient) GetName(ctx context.Context, in *GetName_Request, opts ...grpc.CallOption) (*GetName_Response, error) { + out := new(GetName_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/GetName", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) GetVersion(ctx context.Context, in *GetVersion_Request, opts ...grpc.CallOption) (*GetVersion_Response, error) { + out := new(GetVersion_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/GetVersion", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) GetTables(ctx context.Context, in *GetTables_Request, opts ...grpc.CallOption) (*GetTables_Response, error) { + out := new(GetTables_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/GetTables", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) GetMetrics(ctx context.Context, in *GetMetrics_Request, opts ...grpc.CallOption) (*GetMetrics_Response, error) { + out := new(GetMetrics_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/GetMetrics", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) Init(ctx context.Context, in *Init_Request, opts ...grpc.CallOption) (*Init_Response, error) { + out := new(Init_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/Init", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) GetDynamicTables(ctx context.Context, in *GetDynamicTables_Request, opts ...grpc.CallOption) (*GetDynamicTables_Response, error) { + out := new(GetDynamicTables_Response) + err := c.cc.Invoke(ctx, "/cloudquery.source.v1.Source/GetDynamicTables", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +func (c *sourceClient) Sync(ctx context.Context, in *Sync_Request, opts ...grpc.CallOption) (Source_SyncClient, error) { + stream, err := c.cc.NewStream(ctx, &Source_ServiceDesc.Streams[0], "/cloudquery.source.v1.Source/Sync", opts...) + if err != nil { + return nil, err + } + x := &sourceSyncClient{stream} + if err := x.ClientStream.SendMsg(in); err != nil { + return nil, err + } + if err := x.ClientStream.CloseSend(); err != nil { + return nil, err + } + return x, nil +} + +type Source_SyncClient interface { + Recv() (*Sync_Response, error) + grpc.ClientStream +} + +type sourceSyncClient struct { + grpc.ClientStream +} + +func (x *sourceSyncClient) Recv() (*Sync_Response, error) { + m := new(Sync_Response) + if err := x.ClientStream.RecvMsg(m); err != nil { + return nil, err + } + return m, nil +} + +// SourceServer is the server API for Source service. +// All implementations must embed UnimplementedSourceServer +// for forward compatibility +type SourceServer interface { + // Get the name of the plugin + GetName(context.Context, *GetName_Request) (*GetName_Response, error) + // Get the current version of the plugin + GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) + // Get all static tables the source plugin supports + GetTables(context.Context, *GetTables_Request) (*GetTables_Response, error) + // Get metrics for the source plugin + GetMetrics(context.Context, *GetMetrics_Request) (*GetMetrics_Response, error) + // Configure the plugin with the given credentials and mode + Init(context.Context, *Init_Request) (*Init_Response, error) + // Get all tables the source plugin supports. Must be called after Init + GetDynamicTables(context.Context, *GetDynamicTables_Request) (*GetDynamicTables_Response, error) + // Start the sync the source plugin + Sync(*Sync_Request, Source_SyncServer) error + mustEmbedUnimplementedSourceServer() +} + +// UnimplementedSourceServer must be embedded to have forward compatible implementations. +type UnimplementedSourceServer struct { +} + +func (UnimplementedSourceServer) GetName(context.Context, *GetName_Request) (*GetName_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetName not implemented") +} +func (UnimplementedSourceServer) GetVersion(context.Context, *GetVersion_Request) (*GetVersion_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetVersion not implemented") +} +func (UnimplementedSourceServer) GetTables(context.Context, *GetTables_Request) (*GetTables_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetTables not implemented") +} +func (UnimplementedSourceServer) GetMetrics(context.Context, *GetMetrics_Request) (*GetMetrics_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetMetrics not implemented") +} +func (UnimplementedSourceServer) Init(context.Context, *Init_Request) (*Init_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method Init not implemented") +} +func (UnimplementedSourceServer) GetDynamicTables(context.Context, *GetDynamicTables_Request) (*GetDynamicTables_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetDynamicTables not implemented") +} +func (UnimplementedSourceServer) Sync(*Sync_Request, Source_SyncServer) error { + return status.Errorf(codes.Unimplemented, "method Sync not implemented") +} +func (UnimplementedSourceServer) mustEmbedUnimplementedSourceServer() {} + +// UnsafeSourceServer may be embedded to opt out of forward compatibility for this service. +// Use of this interface is not recommended, as added methods to SourceServer will +// result in compilation errors. +type UnsafeSourceServer interface { + mustEmbedUnimplementedSourceServer() +} + +func RegisterSourceServer(s grpc.ServiceRegistrar, srv SourceServer) { + s.RegisterService(&Source_ServiceDesc, srv) +} + +func _Source_GetName_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetName_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).GetName(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/GetName", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).GetName(ctx, req.(*GetName_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_GetVersion_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetVersion_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).GetVersion(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/GetVersion", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).GetVersion(ctx, req.(*GetVersion_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_GetTables_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetTables_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).GetTables(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/GetTables", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).GetTables(ctx, req.(*GetTables_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_GetMetrics_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetMetrics_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).GetMetrics(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/GetMetrics", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).GetMetrics(ctx, req.(*GetMetrics_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_Init_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(Init_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).Init(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/Init", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).Init(ctx, req.(*Init_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_GetDynamicTables_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetDynamicTables_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(SourceServer).GetDynamicTables(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.source.v1.Source/GetDynamicTables", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(SourceServer).GetDynamicTables(ctx, req.(*GetDynamicTables_Request)) + } + return interceptor(ctx, in, info, handler) +} + +func _Source_Sync_Handler(srv interface{}, stream grpc.ServerStream) error { + m := new(Sync_Request) + if err := stream.RecvMsg(m); err != nil { + return err + } + return srv.(SourceServer).Sync(m, &sourceSyncServer{stream}) +} + +type Source_SyncServer interface { + Send(*Sync_Response) error + grpc.ServerStream +} + +type sourceSyncServer struct { + grpc.ServerStream +} + +func (x *sourceSyncServer) Send(m *Sync_Response) error { + return x.ServerStream.SendMsg(m) +} + +// Source_ServiceDesc is the grpc.ServiceDesc for Source service. +// It's only intended for direct use with grpc.RegisterService, +// and not to be introspected or modified (even as a copy) +var Source_ServiceDesc = grpc.ServiceDesc{ + ServiceName: "cloudquery.source.v1.Source", + HandlerType: (*SourceServer)(nil), + Methods: []grpc.MethodDesc{ + { + MethodName: "GetName", + Handler: _Source_GetName_Handler, + }, + { + MethodName: "GetVersion", + Handler: _Source_GetVersion_Handler, + }, + { + MethodName: "GetTables", + Handler: _Source_GetTables_Handler, + }, + { + MethodName: "GetMetrics", + Handler: _Source_GetMetrics_Handler, + }, + { + MethodName: "Init", + Handler: _Source_Init_Handler, + }, + { + MethodName: "GetDynamicTables", + Handler: _Source_GetDynamicTables_Handler, + }, + }, + Streams: []grpc.StreamDesc{ + { + StreamName: "Sync", + Handler: _Source_Sync_Handler, + ServerStreams: true, + }, + }, + Metadata: "cloudquery/source/v1/source.proto", +} diff --git a/clients/random.go b/internal/random/random.go similarity index 89% rename from clients/random.go rename to internal/random/random.go index 38341eb53d..726e36c9be 100644 --- a/clients/random.go +++ b/internal/random/random.go @@ -1,4 +1,4 @@ -package clients +package random import ( "math/rand" @@ -22,6 +22,6 @@ func randSeq(n int) string { return string(b) } -func generateRandomUnixSocketName() string { +func GenerateRandomUnixSocketName() string { return filepath.Join(unixSocketDir, "cq-"+randSeq(16)+".sock") } diff --git a/internal/servers/destinations.go b/internal/servers/destination/v0/destinations.go similarity index 87% rename from internal/servers/destinations.go rename to internal/servers/destination/v0/destinations.go index c8ade5ae13..fa78a91dfb 100644 --- a/internal/servers/destinations.go +++ b/internal/servers/destination/v0/destinations.go @@ -6,7 +6,8 @@ import ( "fmt" "io" - "github.com/cloudquery/plugin-sdk/internal/pb" + pbBase "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" + pb "github.com/cloudquery/plugin-sdk/internal/pb/destination/v0" "github.com/cloudquery/plugin-sdk/plugins/destination" "github.com/cloudquery/plugin-sdk/schema" "github.com/cloudquery/plugin-sdk/specs" @@ -22,28 +23,28 @@ type DestinationServer struct { Logger zerolog.Logger } -func (*DestinationServer) GetProtocolVersion(context.Context, *pb.GetProtocolVersion_Request) (*pb.GetProtocolVersion_Response, error) { - return &pb.GetProtocolVersion_Response{ +func (*DestinationServer) GetProtocolVersion(context.Context, *pbBase.GetProtocolVersion_Request) (*pbBase.GetProtocolVersion_Response, error) { + return &pbBase.GetProtocolVersion_Response{ Version: 2, }, nil } -func (s *DestinationServer) Configure(ctx context.Context, req *pb.Configure_Request) (*pb.Configure_Response, error) { +func (s *DestinationServer) Configure(ctx context.Context, req *pbBase.Configure_Request) (*pbBase.Configure_Response, error) { var spec specs.Destination if err := json.Unmarshal(req.Config, &spec); err != nil { return nil, status.Errorf(codes.InvalidArgument, "failed to unmarshal spec: %v", err) } - return &pb.Configure_Response{}, s.Plugin.Init(ctx, s.Logger, spec) + return &pbBase.Configure_Response{}, s.Plugin.Init(ctx, s.Logger, spec) } -func (s *DestinationServer) GetName(context.Context, *pb.GetName_Request) (*pb.GetName_Response, error) { - return &pb.GetName_Response{ +func (s *DestinationServer) GetName(context.Context, *pbBase.GetName_Request) (*pbBase.GetName_Response, error) { + return &pbBase.GetName_Response{ Name: s.Plugin.Name(), }, nil } -func (s *DestinationServer) GetVersion(context.Context, *pb.GetVersion_Request) (*pb.GetVersion_Response, error) { - return &pb.GetVersion_Response{ +func (s *DestinationServer) GetVersion(context.Context, *pbBase.GetVersion_Request) (*pbBase.GetVersion_Response, error) { + return &pbBase.GetVersion_Response{ Version: s.Plugin.Version(), }, nil } diff --git a/internal/servers/source.go b/internal/servers/source/v0/source.go similarity index 67% rename from internal/servers/source.go rename to internal/servers/source/v0/source.go index 73e9751662..8bee5c9d41 100644 --- a/internal/servers/source.go +++ b/internal/servers/source/v0/source.go @@ -1,4 +1,4 @@ -package servers +package source import ( "bytes" @@ -7,7 +7,8 @@ import ( "errors" "fmt" - "github.com/cloudquery/plugin-sdk/internal/pb" + pbBase "github.com/cloudquery/plugin-sdk/internal/pb/base/v0" + pb "github.com/cloudquery/plugin-sdk/internal/pb/source/v0" "github.com/cloudquery/plugin-sdk/plugins/source" "github.com/cloudquery/plugin-sdk/schema" "github.com/cloudquery/plugin-sdk/specs" @@ -18,19 +19,20 @@ import ( "google.golang.org/protobuf/proto" ) -type SourceServer struct { +type Server struct { pb.UnimplementedSourceServer Plugin *source.Plugin Logger zerolog.Logger } -func (*SourceServer) GetProtocolVersion(context.Context, *pb.GetProtocolVersion_Request) (*pb.GetProtocolVersion_Response, error) { - return &pb.GetProtocolVersion_Response{ +// Deprecated: use GetSupportedProtocolVersions instead +func (*Server) GetProtocolVersion(context.Context, *pbBase.GetProtocolVersion_Request) (*pbBase.GetProtocolVersion_Response, error) { + return &pbBase.GetProtocolVersion_Response{ Version: 2, }, nil } -func (s *SourceServer) GetTables(context.Context, *pb.GetTables_Request) (*pb.GetTables_Response, error) { +func (s *Server) GetTables(context.Context, *pb.GetTables_Request) (*pb.GetTables_Response, error) { b, err := json.Marshal(s.Plugin.Tables()) if err != nil { return nil, fmt.Errorf("failed to marshal tables: %w", err) @@ -40,7 +42,10 @@ func (s *SourceServer) GetTables(context.Context, *pb.GetTables_Request) (*pb.Ge }, nil } -func (s *SourceServer) GetTablesForSpec(_ context.Context, req *pb.GetTablesForSpec_Request) (*pb.GetTablesForSpec_Response, error) { +func (s *Server) GetTablesForSpec(_ context.Context, req *pb.GetTablesForSpec_Request) (*pb.GetTablesForSpec_Response, error) { + if s.Plugin.HasDynamicTables() { + return nil, errors.New("plugin has dynamic tables, please upgrade CLI version") + } if len(req.Spec) == 0 { b, err := json.Marshal(s.Plugin.Tables()) if err != nil { @@ -70,27 +75,30 @@ func (s *SourceServer) GetTablesForSpec(_ context.Context, req *pb.GetTablesForS }, nil } -func (s *SourceServer) GetName(context.Context, *pb.GetName_Request) (*pb.GetName_Response, error) { - return &pb.GetName_Response{ +func (s *Server) GetName(context.Context, *pbBase.GetName_Request) (*pbBase.GetName_Response, error) { + return &pbBase.GetName_Response{ Name: s.Plugin.Name(), }, nil } -func (s *SourceServer) GetVersion(context.Context, *pb.GetVersion_Request) (*pb.GetVersion_Response, error) { - return &pb.GetVersion_Response{ +func (s *Server) GetVersion(context.Context, *pbBase.GetVersion_Request) (*pbBase.GetVersion_Response, error) { + return &pbBase.GetVersion_Response{ Version: s.Plugin.Version(), }, nil } -func (*SourceServer) GetSyncSummary(context.Context, *pb.GetSyncSummary_Request) (*pb.GetSyncSummary_Response, error) { +func (*Server) GetSyncSummary(context.Context, *pb.GetSyncSummary_Request) (*pb.GetSyncSummary_Response, error) { return nil, status.Errorf(codes.Unimplemented, "method GetSyncSummary is deprecated please upgrade client") } -func (*SourceServer) Sync(*pb.Sync_Request, pb.Source_SyncServer) error { +func (*Server) Sync(*pb.Sync_Request, pb.Source_SyncServer) error { return status.Errorf(codes.Unimplemented, "method Sync is deprecated please upgrade client") } -func (s *SourceServer) Sync2(req *pb.Sync2_Request, stream pb.Source_Sync2Server) error { +func (s *Server) Sync2(req *pb.Sync2_Request, stream pb.Source_Sync2Server) error { + if s.Plugin.HasDynamicTables() { + return errors.New("plugin has dynamic tables, please upgrade CLI version") + } resources := make(chan *schema.Resource) var syncErr error @@ -101,15 +109,23 @@ func (s *SourceServer) Sync2(req *pb.Sync2_Request, stream pb.Source_Sync2Server if err := dec.Decode(&spec); err != nil { return status.Errorf(codes.InvalidArgument, "failed to decode spec: %v", err) } + ctx := stream.Context() + if err := s.Plugin.Init(ctx, spec); err != nil { + return status.Errorf(codes.Internal, "failed to init plugin: %v", err) + } + defer func() { + if err := s.Plugin.Close(ctx); err != nil { + s.Logger.Error().Err(err).Msg("failed to close plugin") + } + }() go func() { defer close(resources) - err := s.Plugin.Sync(stream.Context(), spec, resources) + err := s.Plugin.Sync(ctx, resources) if err != nil { syncErr = fmt.Errorf("failed to sync resources: %w", err) } }() - for resource := range resources { destResource := resource.ToDestinationResource() b, err := json.Marshal(destResource) @@ -135,7 +151,7 @@ func (s *SourceServer) Sync2(req *pb.Sync2_Request, stream pb.Source_Sync2Server return syncErr } -func (s *SourceServer) GetMetrics(context.Context, *pb.GetSourceMetrics_Request) (*pb.GetSourceMetrics_Response, error) { +func (s *Server) GetMetrics(context.Context, *pb.GetSourceMetrics_Request) (*pb.GetSourceMetrics_Response, error) { // Aggregate metrics before sending to keep response size small. // Temporary fix for https://github.com/cloudquery/cloudquery/issues/3962 m := s.Plugin.Metrics() @@ -161,14 +177,14 @@ func (s *SourceServer) GetMetrics(context.Context, *pb.GetSourceMetrics_Request) func checkMessageSize(msg proto.Message, resource *schema.Resource) error { size := proto.Size(msg) // log error to Sentry if row exceeds half of the max size - if size > MaxMsgSize/2 { + if size > pb.MaxMsgSize/2 { sentry.WithScope(func(scope *sentry.Scope) { scope.SetTag("table", resource.Table.Name) scope.SetExtra("bytes", size) sentry.CurrentHub().CaptureMessage("Large message detected") }) } - if size > MaxMsgSize { + if size > pb.MaxMsgSize { return errors.New("message exceeds max size") } return nil diff --git a/internal/servers/source/v1/source.go b/internal/servers/source/v1/source.go new file mode 100644 index 0000000000..3133da2964 --- /dev/null +++ b/internal/servers/source/v1/source.go @@ -0,0 +1,150 @@ +package source + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + + pb "github.com/cloudquery/plugin-sdk/internal/pb/source/v1" + "github.com/cloudquery/plugin-sdk/plugins/source" + "github.com/cloudquery/plugin-sdk/schema" + "github.com/cloudquery/plugin-sdk/specs" + "github.com/getsentry/sentry-go" + "github.com/rs/zerolog" + "google.golang.org/grpc/codes" + "google.golang.org/grpc/status" + "google.golang.org/protobuf/proto" +) + +type Server struct { + pb.UnimplementedSourceServer + Plugin *source.Plugin + Logger zerolog.Logger +} + +func (s *Server) GetTables(context.Context, *pb.GetTables_Request) (*pb.GetTables_Response, error) { + b, err := json.Marshal(s.Plugin.Tables()) + if err != nil { + return nil, fmt.Errorf("failed to marshal tables: %w", err) + } + return &pb.GetTables_Response{ + Tables: b, + }, nil +} + +func (s *Server) GetDynamicTables(context.Context, *pb.GetDynamicTables_Request) (*pb.GetDynamicTables_Response, error) { + tables := s.Plugin.GetDynamicTables() + b, err := json.Marshal(tables) + if err != nil { + return nil, fmt.Errorf("failed to marshal tables: %w", err) + } + return &pb.GetDynamicTables_Response{ + Tables: b, + }, nil +} + +func (s *Server) GetName(context.Context, *pb.GetName_Request) (*pb.GetName_Response, error) { + return &pb.GetName_Response{ + Name: s.Plugin.Name(), + }, nil +} + +func (s *Server) GetVersion(context.Context, *pb.GetVersion_Request) (*pb.GetVersion_Response, error) { + return &pb.GetVersion_Response{ + Version: s.Plugin.Version(), + }, nil +} + +func (s *Server) Init(ctx context.Context, req *pb.Init_Request) (*pb.Init_Response, error) { + var spec specs.Source + dec := json.NewDecoder(bytes.NewReader(req.Spec)) + dec.UseNumber() + // TODO: warn about unknown fields + if err := dec.Decode(&spec); err != nil { + return nil, status.Errorf(codes.InvalidArgument, "failed to decode spec: %v", err) + } + + if err := s.Plugin.Init(ctx, spec); err != nil { + return nil, status.Errorf(codes.Internal, "failed to init plugin: %v", err) + } + return &pb.Init_Response{}, nil +} + +func (s *Server) Sync(_ *pb.Sync_Request, stream pb.Source_SyncServer) error { + resources := make(chan *schema.Resource) + var syncErr error + ctx := stream.Context() + + go func() { + defer close(resources) + err := s.Plugin.Sync(ctx, resources) + if err != nil { + syncErr = fmt.Errorf("failed to sync resources: %w", err) + } + }() + + for resource := range resources { + destResource := resource.ToDestinationResource() + b, err := json.Marshal(destResource) + if err != nil { + return status.Errorf(codes.Internal, "failed to marshal resource: %v", err) + } + + msg := &pb.Sync_Response{ + Resource: b, + } + err = checkMessageSize(msg, resource) + if err != nil { + s.Logger.Warn().Str("table", resource.Table.Name). + Int("bytes", len(msg.String())). + Msg("Row exceeding max bytes ignored") + continue + } + if err := stream.Send(msg); err != nil { + return status.Errorf(codes.Internal, "failed to send resource: %v", err) + } + } + + return syncErr +} + +func (s *Server) GetMetrics(context.Context, *pb.GetMetrics_Request) (*pb.GetMetrics_Response, error) { + // Aggregate metrics before sending to keep response size small. + // Temporary fix for https://github.com/cloudquery/cloudquery/issues/3962 + m := s.Plugin.Metrics() + agg := &source.TableClientMetrics{} + for _, table := range m.TableClient { + for _, tableClient := range table { + agg.Resources += tableClient.Resources + agg.Errors += tableClient.Errors + agg.Panics += tableClient.Panics + } + } + b, err := json.Marshal(&source.Metrics{ + TableClient: map[string]map[string]*source.TableClientMetrics{"": {"": agg}}, + }) + if err != nil { + return nil, fmt.Errorf("failed to marshal source metrics: %w", err) + } + return &pb.GetMetrics_Response{ + Metrics: b, + }, nil +} + +func checkMessageSize(msg proto.Message, resource *schema.Resource) error { + size := proto.Size(msg) + // log error to Sentry if row exceeds half of the max size + if size > pb.MaxMsgSize/2 { + sentry.WithScope(func(scope *sentry.Scope) { + scope.SetTag("table", resource.Table.Name) + scope.SetExtra("bytes", size) + sentry.CurrentHub().CaptureMessage("Large message detected") + }) + } + if size > pb.MaxMsgSize { + return errors.New("message exceeds max size") + } + return nil +} diff --git a/plugins/destination/plugin.go b/plugins/destination/plugin.go index 388f3c8acf..a2ace6c403 100644 --- a/plugins/destination/plugin.go +++ b/plugins/destination/plugin.go @@ -256,8 +256,12 @@ func (p *Plugin) Write(ctx context.Context, sourceSpec specs.Source, tables sche if p.spec.WriteMode == specs.WriteModeOverwriteDeleteStale { tablesToDelete := tables if sourceSpec.Backend != specs.BackendNone { - include := func(t *schema.Table) bool { return true } - exclude := func(t *schema.Table) bool { return t.IsIncremental } + include := func(t *schema.Table) bool { + return true + } + exclude := func(t *schema.Table) bool { + return t.IsIncremental + } tablesToDelete = tables.FilterDfsFunc(include, exclude) } if err := p.DeleteStale(ctx, tablesToDelete, sourceSpec.Name, syncTime); err != nil { diff --git a/plugins/source/benchmark_test.go b/plugins/source/benchmark_test.go index 895cfc961c..955d21474b 100644 --- a/plugins/source/benchmark_test.go +++ b/plugins/source/benchmark_test.go @@ -195,12 +195,14 @@ func (s *Benchmark) Run() { Concurrency: s.Concurrency, Scheduler: s.Scheduler, } + if err := s.plugin.Init(ctx, spec); err != nil { + s.b.Fatal(err) + } resources := make(chan *schema.Resource) g, ctx := errgroup.WithContext(ctx) g.Go(func() error { defer close(resources) return s.plugin.Sync(ctx, - spec, resources) }) s.b.StartTimer() diff --git a/plugins/source/options.go b/plugins/source/options.go new file mode 100644 index 0000000000..f701ce9225 --- /dev/null +++ b/plugins/source/options.go @@ -0,0 +1,18 @@ +package source + +import ( + "context" + + "github.com/cloudquery/plugin-sdk/schema" +) + +type GetTables func(ctx context.Context, c schema.ClientMeta) (schema.Tables, error) + +type Option func(*Plugin) + +// WithDynamicTableOption allows the plugin to return list of tables after call to New +func WithDynamicTableOption(getDynamicTables GetTables) Option { + return func(p *Plugin) { + p.getDynamicTables = getDynamicTables + } +} diff --git a/plugins/source/plugin.go b/plugins/source/plugin.go index 820002ab60..66b630265c 100644 --- a/plugins/source/plugin.go +++ b/plugins/source/plugin.go @@ -3,6 +3,7 @@ package source import ( "context" "fmt" + "sync" "time" "github.com/cloudquery/plugin-sdk/backend" @@ -29,6 +30,8 @@ type Plugin struct { version string // Called upon configure call to validate and init configuration newExecutionClient NewExecutionClientFunc + // dynamic table function if specified + getDynamicTables GetTables // Tables is all tables supported by this source plugin tables schema.Tables // status sync metrics @@ -43,6 +46,17 @@ type Plugin struct { maxDepth uint64 // caser caser *caser.Caser + // mu is a mutex that limits the number of concurrent init/syncs (can only be one at a time) + mu sync.Mutex + + // client is the initialized session client + client schema.ClientMeta + // sessionTables are the + sessionTables schema.Tables + // backend is the backend used to store the cursor state + backend backend.Backend + // spec is the spec the client was initialized with + spec specs.Source } const ( @@ -50,18 +64,21 @@ const ( ) // Add internal columns -func addInternalColumns(tables []*schema.Table) { +func addInternalColumns(tables []*schema.Table) error { for _, table := range tables { if c := table.Column("_cq_id"); c != nil { - panic(fmt.Sprintf("table %s already has column _cq_id", table.Name)) + return fmt.Errorf("table %s already has column _cq_id", table.Name) } cqID := schema.CqIDColumn if len(table.PrimaryKeys()) == 0 { cqID.CreationOptions.PrimaryKey = true } table.Columns = append([]schema.Column{cqID, schema.CqParentIDColumn}, table.Columns...) - addInternalColumns(table.Relations) + if err := addInternalColumns(table.Relations); err != nil { + return err + } } + return nil } // Set parent links on relational tables @@ -103,7 +120,7 @@ func maxDepth(tables schema.Tables) uint64 { // NewPlugin returns a new plugin with a given name, version, tables, newExecutionClient // and additional options. -func NewPlugin(name string, version string, tables []*schema.Table, newExecutionClient NewExecutionClientFunc) *Plugin { +func NewPlugin(name string, version string, tables []*schema.Table, newExecutionClient NewExecutionClientFunc, options ...Option) *Plugin { p := Plugin{ name: name, version: version, @@ -112,11 +129,16 @@ func NewPlugin(name string, version string, tables []*schema.Table, newExecution metrics: &Metrics{TableClient: make(map[string]map[string]*TableClientMetrics)}, caser: caser.New(), } + for _, opt := range options { + opt(&p) + } setParents(p.tables, nil) if err := transformTables(p.tables); err != nil { panic(err) } - addInternalColumns(p.tables) + if err := addInternalColumns(p.tables); err != nil { + panic(err) + } if err := p.validate(); err != nil { panic(err) } @@ -136,8 +158,17 @@ func (p *Plugin) Tables() schema.Tables { return p.tables } +func (p *Plugin) HasDynamicTables() bool { + return p.getDynamicTables != nil +} + +func (p *Plugin) GetDynamicTables() schema.Tables { + return p.sessionTables +} + // TablesForSpec returns all tables supported by this source plugin that match the given spec. // It validates the tables part of the spec and will return an error if it is found to be invalid. +// This is deprecated method func (p *Plugin) TablesForSpec(spec specs.Source) (schema.Tables, error) { spec.SetDefaults() if err := spec.Validate(); err != nil { @@ -164,27 +195,24 @@ func (p *Plugin) Metrics() *Metrics { return p.metrics } -// Sync is syncing data from the requested tables in spec to the given channel -func (p *Plugin) Sync(ctx context.Context, spec specs.Source, res chan<- *schema.Resource) error { +func (p *Plugin) Init(ctx context.Context, spec specs.Source) error { + if !p.mu.TryLock() { + return fmt.Errorf("plugin already in use") + } + defer p.mu.Unlock() + + var err error spec.SetDefaults() if err := spec.Validate(); err != nil { return fmt.Errorf("invalid spec: %w", err) } - tables, err := p.tables.FilterDfs(spec.Tables, spec.SkipTables) - if err != nil { - return fmt.Errorf("failed to filter tables: %w", err) - } + p.spec = spec - if len(tables) == 0 { - return fmt.Errorf("no tables to sync - please check your spec 'tables' and 'skip_tables' settings") - } - - var be backend.Backend switch spec.Backend { case specs.BackendNone: // do nothing case specs.BackendLocal: - be, err = local.New(spec) + p.backend, err = local.New(spec) if err != nil { return fmt.Errorf("failed to initialize local backend: %w", err) } @@ -192,30 +220,82 @@ func (p *Plugin) Sync(ctx context.Context, spec specs.Source, res chan<- *schema return fmt.Errorf("unknown backend: %s", spec.Backend) } - if be != nil { - defer func() { - p.logger.Info().Msg("closing backend") - err := be.Close(ctx) - if err != nil { - p.logger.Error().Err(err).Msg("failed to close backend") - } - }() - } - - c, err := p.newExecutionClient(ctx, p.logger, spec, Options{Backend: be}) + p.client, err = p.newExecutionClient(ctx, p.logger, spec, Options{Backend: p.backend}) if err != nil { return fmt.Errorf("failed to create execution client for source plugin %s: %w", p.name, err) } + tables := p.tables + if p.getDynamicTables != nil { + tables, err = p.getDynamicTables(ctx, p.client) + if err != nil { + return fmt.Errorf("failed to get dynamic tables: %w", err) + } + + tables, err = tables.FilterDfs(spec.Tables, spec.SkipTables) + if err != nil { + return fmt.Errorf("failed to filter tables: %w", err) + } + if len(tables) == 0 { + return fmt.Errorf("no tables to sync - please check your spec 'tables' and 'skip_tables' settings") + } + + setParents(tables, nil) + if err := transformTables(tables); err != nil { + return err + } + if err := addInternalColumns(tables); err != nil { + return err + } + if err := p.validate(); err != nil { + return err + } + p.maxDepth = maxDepth(tables) + if p.maxDepth > maxAllowedDepth { + return fmt.Errorf("max depth of tables is %d, max allowed is %d", p.maxDepth, maxAllowedDepth) + } + } else { + tables, err = tables.FilterDfs(spec.Tables, spec.SkipTables) + if err != nil { + return fmt.Errorf("failed to filter tables: %w", err) + } + } + + p.sessionTables = tables + return nil +} + +// Sync is syncing data from the requested tables in spec to the given channel +func (p *Plugin) Sync(ctx context.Context, res chan<- *schema.Resource) error { + if !p.mu.TryLock() { + return fmt.Errorf("plugin already in use") + } + defer p.mu.Unlock() + startTime := time.Now() - switch spec.Scheduler { + switch p.spec.Scheduler { case specs.SchedulerDFS: - p.syncDfs(ctx, spec, c, tables, res) + p.syncDfs(ctx, p.spec, p.client, p.sessionTables, res) case specs.SchedulerRoundRobin: - p.syncRoundRobin(ctx, spec, c, tables, res) + p.syncRoundRobin(ctx, p.spec, p.client, p.sessionTables, res) default: - return fmt.Errorf("unknown scheduler %s. Options are: %v", spec.Scheduler, specs.AllSchedulers.String()) + return fmt.Errorf("unknown scheduler %s. Options are: %v", p.spec.Scheduler, specs.AllSchedulers.String()) } p.logger.Info().Uint64("resources", p.metrics.TotalResources()).Uint64("errors", p.metrics.TotalErrors()).Uint64("panics", p.metrics.TotalPanics()).TimeDiff("duration", time.Now(), startTime).Msg("sync finished") return nil } + +func (p *Plugin) Close(ctx context.Context) error { + if !p.mu.TryLock() { + return fmt.Errorf("plugin already in use") + } + defer p.mu.Unlock() + if p.backend != nil { + err := p.backend.Close(ctx) + if err != nil { + return fmt.Errorf("failed to close backend: %w", err) + } + p.backend = nil + } + return nil +} diff --git a/plugins/source/plugin_test.go b/plugins/source/plugin_test.go index 402be5623a..657a148d3e 100644 --- a/plugins/source/plugin_test.go +++ b/plugins/source/plugin_test.go @@ -269,12 +269,15 @@ func testSyncTable(t *testing.T, tc syncTestCase, scheduler specs.Scheduler) { Concurrency: 1, // choose a very low value to check that we don't run into deadlocks Scheduler: scheduler, } + if err := plugin.Init(ctx, spec); err != nil { + t.Fatal(err) + } + resources := make(chan *schema.Resource) g, ctx := errgroup.WithContext(ctx) g.Go(func() error { defer close(resources) return plugin.Sync(ctx, - spec, resources) }) diff --git a/plugins/source/testing.go b/plugins/source/testing.go index 96302b8148..8aa1e99215 100644 --- a/plugins/source/testing.go +++ b/plugins/source/testing.go @@ -24,9 +24,13 @@ func TestPluginSync(t *testing.T, plugin *Plugin, spec specs.Source, opts ...Tes resourcesChannel := make(chan *schema.Resource) var syncErr error + if err := plugin.Init(context.Background(), spec); err != nil { + t.Fatal(err) + } + go func() { defer close(resourcesChannel) - syncErr = plugin.Sync(context.Background(), spec, resourcesChannel) + syncErr = plugin.Sync(context.Background(), resourcesChannel) }() syncedResources := make([]*schema.Resource, 0) diff --git a/clients/download.go b/registry/download.go similarity index 97% rename from clients/download.go rename to registry/download.go index b1063b33ba..792caf091e 100644 --- a/clients/download.go +++ b/registry/download.go @@ -1,4 +1,4 @@ -package clients +package registry import ( "archive/zip" @@ -59,7 +59,7 @@ func DownloadPluginFromGithub(ctx context.Context, localPath string, org string, if org != "cloudquery" { pathInArchive = fmt.Sprintf("cq-%s-%s", typ, name) } - pathInArchive = withBinarySuffix(pathInArchive) + pathInArchive = WithBinarySuffix(pathInArchive) fileInArchive, err := archive.Open(pathInArchive) if err != nil { @@ -156,7 +156,7 @@ func downloadProgressBar(maxBytes int64, description ...string) *progressbar.Pro ) } -func withBinarySuffix(filePath string) string { +func WithBinarySuffix(filePath string) string { if runtime.GOOS == "windows" { return filePath + ".exe" } diff --git a/serve/destination.go b/serve/destination.go index 257c2a3366..2489ca457d 100644 --- a/serve/destination.go +++ b/serve/destination.go @@ -8,8 +8,8 @@ import ( "strings" "sync" - "github.com/cloudquery/plugin-sdk/internal/pb" - "github.com/cloudquery/plugin-sdk/internal/servers" + pbv0 "github.com/cloudquery/plugin-sdk/internal/pb/destination/v0" + servers "github.com/cloudquery/plugin-sdk/internal/servers/destination/v0" "github.com/cloudquery/plugin-sdk/plugins/destination" "github.com/getsentry/sentry-go" grpczerolog "github.com/grpc-ecosystem/go-grpc-middleware/providers/zerolog/v2" @@ -105,10 +105,10 @@ func newCmdDestinationServe(serve *destinationServe) *cobra.Command { grpc.ChainStreamInterceptor( logging.StreamServerInterceptor(grpczerolog.InterceptorLogger(logger)), ), - grpc.MaxRecvMsgSize(servers.MaxMsgSize), - grpc.MaxSendMsgSize(servers.MaxMsgSize), + grpc.MaxRecvMsgSize(pbv0.MaxMsgSize), + grpc.MaxSendMsgSize(pbv0.MaxMsgSize), ) - pb.RegisterDestinationServer(s, &servers.DestinationServer{ + pbv0.RegisterDestinationServer(s, &servers.DestinationServer{ Plugin: serve.plugin, Logger: logger, }) diff --git a/serve/destination_test.go b/serve/destination_v0_test.go similarity index 94% rename from serve/destination_test.go rename to serve/destination_v0_test.go index 6b27bf88bb..77dcfc2de3 100644 --- a/serve/destination_test.go +++ b/serve/destination_v0_test.go @@ -8,7 +8,7 @@ import ( "testing" "time" - "github.com/cloudquery/plugin-sdk/clients" + clients "github.com/cloudquery/plugin-sdk/clients/destination/v0" "github.com/cloudquery/plugin-sdk/internal/memdb" "github.com/cloudquery/plugin-sdk/plugins/destination" "github.com/cloudquery/plugin-sdk/schema" @@ -62,7 +62,7 @@ func TestDestination(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewDestinationClient(ctx, specs.RegistryGrpc, "", "", clients.WithDestinationGrpcConn(conn), clients.WithDestinationNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithDestinationGrpcConn(conn), clients.WithDestinationNoSentry()) if err != nil { t.Fatal(err) } diff --git a/serve/source.go b/serve/source.go index d7ef87aac8..13cfca9aa4 100644 --- a/serve/source.go +++ b/serve/source.go @@ -8,8 +8,10 @@ import ( "strings" "sync" - "github.com/cloudquery/plugin-sdk/internal/pb" - "github.com/cloudquery/plugin-sdk/internal/servers" + pbv0 "github.com/cloudquery/plugin-sdk/internal/pb/source/v0" + pbv1 "github.com/cloudquery/plugin-sdk/internal/pb/source/v1" + serversv0 "github.com/cloudquery/plugin-sdk/internal/servers/source/v0" + serversv1 "github.com/cloudquery/plugin-sdk/internal/servers/source/v1" "github.com/cloudquery/plugin-sdk/plugins/source" "github.com/getsentry/sentry-go" grpczerolog "github.com/grpc-ecosystem/go-grpc-middleware/providers/zerolog/v2" @@ -111,11 +113,15 @@ func newCmdSourceServe(serve *sourceServe) *cobra.Command { grpc.ChainStreamInterceptor( logging.StreamServerInterceptor(grpczerolog.InterceptorLogger(logger)), ), - grpc.MaxRecvMsgSize(servers.MaxMsgSize), - grpc.MaxSendMsgSize(servers.MaxMsgSize), + grpc.MaxRecvMsgSize(pbv1.MaxMsgSize), + grpc.MaxSendMsgSize(pbv1.MaxMsgSize), ) serve.plugin.SetLogger(logger) - pb.RegisterSourceServer(s, &servers.SourceServer{ + pbv0.RegisterSourceServer(s, &serversv0.Server{ + Plugin: serve.plugin, + Logger: logger, + }) + pbv1.RegisterSourceServer(s, &serversv1.Server{ Plugin: serve.plugin, Logger: logger, }) diff --git a/serve/source_v0_test.go b/serve/source_v0_test.go new file mode 100644 index 0000000000..6e605ccd64 --- /dev/null +++ b/serve/source_v0_test.go @@ -0,0 +1,156 @@ +package serve + +import ( + "context" + "encoding/json" + "fmt" + "sync" + "testing" + "time" + + clients "github.com/cloudquery/plugin-sdk/clients/source/v1" + "github.com/cloudquery/plugin-sdk/plugins/source" + "github.com/cloudquery/plugin-sdk/schema" + "github.com/cloudquery/plugin-sdk/specs" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" +) + +func TestSourceSuccessV1(t *testing.T) { + plugin := source.NewPlugin( + "testPlugin", + "v1.0.0", + []*schema.Table{testTable("test_table"), testTable("test_table2")}, + newTestExecutionClient) + + cmd := newCmdSourceRoot(&sourceServe{ + plugin: plugin, + }) + cmd.SetArgs([]string{"serve", "--network", "test"}) + ctx := context.Background() + ctx, cancel := context.WithCancel(ctx) + var wg sync.WaitGroup + wg.Add(1) + var serverErr error + go func() { + defer wg.Done() + serverErr = cmd.ExecuteContext(ctx) + }() + defer func() { + cancel() + wg.Wait() + }() + for { + testSourceListenerLock.Lock() + if testSourceListener != nil { + testSourceListenerLock.Unlock() + break + } + testSourceListenerLock.Unlock() + t.Log("waiting for grpc server to start") + time.Sleep(time.Millisecond * 200) + } + + // https://stackoverflow.com/questions/42102496/testing-a-grpc-service + conn, err := grpc.DialContext(ctx, "bufnet", grpc.WithContextDialer(bufSourceDialer), grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithBlock()) + if err != nil { + t.Fatalf("Failed to dial bufnet: %v", err) + } + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + if err != nil { + t.Fatal(err) + } + defer func() { + if err := c.Terminate(); err != nil { + t.Fatal(err) + } + }() + + name, err := c.Name(ctx) + if err != nil { + t.Fatal(err) + } + if name != "testPlugin" { + t.Fatalf("expected name to be testPlugin but got %s", name) + } + + version, err := c.Version(ctx) + if err != nil { + t.Fatal(err) + } + if version != "v1.0.0" { + t.Fatalf("Expected version to be v1.0.0 but got %s", version) + } + + tables, err := c.GetTables(ctx) + if err != nil { + t.Fatal(err) + } + if len(tables) != 2 { + t.Fatalf("Expected 2 tables but got %d", len(tables)) + } + + if err := c.Init(ctx, specs.Source{ + Name: "testSourcePlugin", + Version: "v1.0.0", + Path: "cloudquery/testSourcePlugin", + Registry: specs.RegistryGithub, + Tables: []string{"test_table"}, + Spec: TestSourcePluginSpec{Accounts: []string{"cloudquery/plugin-sdk"}}, + Destinations: []string{"test"}, + }); err != nil { + t.Fatal(err) + } + + resources := make(chan []byte, 2) + if err := c.Sync(ctx, + resources); err != nil { + t.Fatal(err) + } + close(resources) + + totalResources := 0 + for resourceB := range resources { + var resource schema.DestinationResource + if err := json.Unmarshal(resourceB, &resource); err != nil { + t.Fatalf("failed to unmarshal resource: %v", err) + } + if resource.TableName != "test_table" { + t.Fatalf("Expected resource with table name test_table. got: %s", resource.TableName) + } + if len(resource.Data) != 3 { + t.Fatalf("Expected resource with data length 3 but got %d", len(resource.Data)) + } + fmt.Println(resource.Data) + if resource.Data[2] == nil { + t.Fatalf("Expected resource with data[2] to be not nil") + } + totalResources++ + } + if totalResources != 1 { + t.Fatalf("Expected 1 resource on channel but got %d", totalResources) + } + + stats, err := c.GetMetrics(ctx) + if err != nil { + t.Fatal(err) + } + clientStats := stats.TableClient[""][""] + if clientStats.Resources != 1 { + t.Fatalf("Expected 1 resource but got %d", clientStats.Resources) + } + + if clientStats.Errors != 0 { + t.Fatalf("Expected 0 errors but got %d", clientStats.Errors) + } + + if clientStats.Panics != 0 { + t.Fatalf("Expected 0 panics but got %d", clientStats.Panics) + } + + cancel() + wg.Wait() + if serverErr != nil { + t.Fatal(serverErr) + } +} diff --git a/serve/source_test.go b/serve/source_v1_test.go similarity index 89% rename from serve/source_test.go rename to serve/source_v1_test.go index 767f17e031..1a25d03b0c 100644 --- a/serve/source_test.go +++ b/serve/source_v1_test.go @@ -9,7 +9,7 @@ import ( "testing" "time" - "github.com/cloudquery/plugin-sdk/clients" + clients "github.com/cloudquery/plugin-sdk/clients/source/v0" "github.com/cloudquery/plugin-sdk/plugins/source" "github.com/cloudquery/plugin-sdk/schema" "github.com/cloudquery/plugin-sdk/specs" @@ -115,7 +115,7 @@ func TestSourceSuccess(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewSourceClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) if err != nil { t.Fatal(err) } @@ -164,8 +164,7 @@ func TestSourceSuccess(t *testing.T) { if len(tables) != 1 { t.Fatalf("Expected 1 table but got %d", len(tables)) } - - resources := make(chan []byte, 1) + resources := make(chan []byte, 2) if err := c.Sync2(ctx, specs.Source{ Name: "testSourcePlugin", @@ -197,9 +196,6 @@ func TestSourceSuccess(t *testing.T) { if resource.Data[2] == nil { t.Fatalf("Expected resource with data[2] to be not nil") } - // if resource.Data[2].Type() != schema.TypeInt { - // t.Fatalf("Expected resource with data type int but got %s", resource.Data[2].Type()) - // } totalResources++ } if totalResources != 1 { @@ -230,8 +226,6 @@ func TestSourceSuccess(t *testing.T) { } } -const testSourceFailExpectedErr = "failed to fetch resources from stream: rpc error: code = Unknown desc = failed to sync resources: failed to create execution client for source plugin testSourcePlugin: error in newTestExecutionClientErr" - func TestSourceFail(t *testing.T) { plugin := source.NewPlugin( "testSourcePlugin", @@ -272,7 +266,7 @@ func TestSourceFail(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewSourceClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) if err != nil { t.Fatal(err) } @@ -299,9 +293,6 @@ func TestSourceFail(t *testing.T) { t.Fatal("expected error but got nil") } - if err.Error() != testSourceFailExpectedErr { - t.Fatalf("expected error %s but got %v", testSourceFailExpectedErr, err) - } cancel() wg.Wait() if serverErr != nil { From 38b136b9aa15dc049f9b66dcd4ceca60fa7bdca6 Mon Sep 17 00:00:00 2001 From: Herman Schaaf Date: Wed, 18 Jan 2023 11:17:15 +0000 Subject: [PATCH 2/5] fix: Simplify client naming conventions (#617) Import paths and references will need to be updated after https://github.com/cloudquery/plugin-sdk/pull/610 anyway, let's use this opportunity to change from e.g. `destination.WithDestinationLogger(log.Logger)` to `destination.WithLogger(log.Logger)` ~I will update the tests momentarily...~ done --- clients/destination/v0/destination.go | 8 ++++---- clients/destination/v0/destination_test.go | 4 ++-- clients/source/v0/source.go | 8 ++++---- clients/source/v0/source_test.go | 2 +- clients/source/v1/source.go | 8 ++++---- serve/destination_v0_test.go | 2 +- serve/source_v0_test.go | 2 +- serve/source_v1_test.go | 4 ++-- 8 files changed, 19 insertions(+), 19 deletions(-) diff --git a/clients/destination/v0/destination.go b/clients/destination/v0/destination.go index b37571a9d9..cc3cf1f420 100644 --- a/clients/destination/v0/destination.go +++ b/clients/destination/v0/destination.go @@ -44,26 +44,26 @@ type Client struct { type ClientOption func(*Client) -func WithDestinationLogger(logger zerolog.Logger) func(*Client) { +func WithLogger(logger zerolog.Logger) func(*Client) { return func(c *Client) { c.logger = logger } } -func WithDestinationDirectory(directory string) func(*Client) { +func WithDirectory(directory string) func(*Client) { return func(c *Client) { c.directory = directory } } -func WithDestinationGrpcConn(userConn *grpc.ClientConn) func(*Client) { +func WithGrpcConn(userConn *grpc.ClientConn) func(*Client) { return func(c *Client) { // we use a different variable here because we don't want to close a connection that wasn't created by us. c.userConn = userConn } } -func WithDestinationNoSentry() func(*Client) { +func WithNoSentry() func(*Client) { return func(c *Client) { c.noSentry = true } diff --git a/clients/destination/v0/destination_test.go b/clients/destination/v0/destination_test.go index df84afd028..0916d5feb7 100644 --- a/clients/destination/v0/destination_test.go +++ b/clients/destination/v0/destination_test.go @@ -38,7 +38,7 @@ func TestDestinationClient(t *testing.T) { for _, tc := range newDestinationClientTestCases { t.Run(tc.Path+"_"+tc.Version, func(t *testing.T) { dirName := t.TempDir() - c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithDestinationLogger(l), WithDestinationDirectory(dirName)) + c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithLogger(l), WithDirectory(dirName)) if err != nil { if strings.HasPrefix(err.Error(), "destination plugin protocol version") { // this also means success as in this tests we just want to make sure we were able to download and spawn the plugin @@ -69,7 +69,7 @@ func TestDestinationClientWriteReturnsCorrectError(t *testing.T) { ctx := context.Background() l := zerolog.New(zerolog.NewTestWriter(t)).Output(zerolog.ConsoleWriter{Out: os.Stderr}).Level(zerolog.DebugLevel) dirName := t.TempDir() - c, err := NewClient(ctx, specs.RegistryGithub, "cloudquery/sqlite", "v1.0.11", WithDestinationLogger(l), WithDestinationDirectory(dirName)) + c, err := NewClient(ctx, specs.RegistryGithub, "cloudquery/sqlite", "v1.0.11", WithLogger(l), WithDirectory(dirName)) if err != nil { t.Fatal(err) } diff --git a/clients/source/v0/source.go b/clients/source/v0/source.go index 14ab156549..aed302db46 100644 --- a/clients/source/v0/source.go +++ b/clients/source/v0/source.go @@ -47,26 +47,26 @@ type FetchResultMessage struct { type ClientOption func(*Client) -func WithSourceLogger(logger zerolog.Logger) func(*Client) { +func WithLogger(logger zerolog.Logger) func(*Client) { return func(c *Client) { c.logger = logger } } -func WithSourceDirectory(directory string) func(*Client) { +func WithDirectory(directory string) func(*Client) { return func(c *Client) { c.directory = directory } } -func WithSourceGRPCConnection(userConn *grpc.ClientConn) func(*Client) { +func WithGRPCConnection(userConn *grpc.ClientConn) func(*Client) { return func(c *Client) { // we use a different variable here because we don't want to close a connection that wasn't created by us. c.userConn = userConn } } -func WithSourceNoSentry() func(*Client) { +func WithNoSentry() func(*Client) { return func(c *Client) { c.noSentry = true } diff --git a/clients/source/v0/source_test.go b/clients/source/v0/source_test.go index ad8985760f..e87e7a4253 100644 --- a/clients/source/v0/source_test.go +++ b/clients/source/v0/source_test.go @@ -33,7 +33,7 @@ func TestSourceClient(t *testing.T) { for _, tc := range newSourceClientTestCases { t.Run(tc.Path+"_"+tc.Version, func(t *testing.T) { dirName := t.TempDir() - c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithSourceLogger(l), WithSourceDirectory(dirName)) + c, err := NewClient(ctx, tc.Registry, tc.Path, tc.Version, WithLogger(l), WithDirectory(dirName)) if err != nil { if strings.HasPrefix(err.Error(), "source plugin protocol version") { // this also means success as in this tests we just want to make sure we were able to download and spawn the plugin diff --git a/clients/source/v1/source.go b/clients/source/v1/source.go index a7dc710882..0ab4aa71cb 100644 --- a/clients/source/v1/source.go +++ b/clients/source/v1/source.go @@ -44,26 +44,26 @@ type FetchResultMessage struct { type ClientOption func(*Client) -func WithSourceLogger(logger zerolog.Logger) func(*Client) { +func WithLogger(logger zerolog.Logger) func(*Client) { return func(c *Client) { c.logger = logger } } -func WithSourceDirectory(directory string) func(*Client) { +func WithDirectory(directory string) func(*Client) { return func(c *Client) { c.directory = directory } } -func WithSourceGRPCConnection(userConn *grpc.ClientConn) func(*Client) { +func WithGRPCConnection(userConn *grpc.ClientConn) func(*Client) { return func(c *Client) { // we use a different variable here because we don't want to close a connection that wasn't created by us. c.userConn = userConn } } -func WithSourceNoSentry() func(*Client) { +func WithNoSentry() func(*Client) { return func(c *Client) { c.noSentry = true } diff --git a/serve/destination_v0_test.go b/serve/destination_v0_test.go index 77dcfc2de3..c0991f4f39 100644 --- a/serve/destination_v0_test.go +++ b/serve/destination_v0_test.go @@ -62,7 +62,7 @@ func TestDestination(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithDestinationGrpcConn(conn), clients.WithDestinationNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithGrpcConn(conn), clients.WithNoSentry()) if err != nil { t.Fatal(err) } diff --git a/serve/source_v0_test.go b/serve/source_v0_test.go index 6e605ccd64..e8dc11bca4 100644 --- a/serve/source_v0_test.go +++ b/serve/source_v0_test.go @@ -56,7 +56,7 @@ func TestSourceSuccessV1(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithGRPCConnection(conn), clients.WithNoSentry()) if err != nil { t.Fatal(err) } diff --git a/serve/source_v1_test.go b/serve/source_v1_test.go index 1a25d03b0c..69a5b01f23 100644 --- a/serve/source_v1_test.go +++ b/serve/source_v1_test.go @@ -115,7 +115,7 @@ func TestSourceSuccess(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithGRPCConnection(conn), clients.WithNoSentry()) if err != nil { t.Fatal(err) } @@ -266,7 +266,7 @@ func TestSourceFail(t *testing.T) { if err != nil { t.Fatalf("Failed to dial bufnet: %v", err) } - c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithSourceGRPCConnection(conn), clients.WithSourceNoSentry()) + c, err := clients.NewClient(ctx, specs.RegistryGrpc, "", "", clients.WithGRPCConnection(conn), clients.WithNoSentry()) if err != nil { t.Fatal(err) } From 0d3ff48d4a8ce324b5685c3df9196943d09b2eba Mon Sep 17 00:00:00 2001 From: Erez Rokah Date: Wed, 18 Jan 2023 15:26:36 +0200 Subject: [PATCH 3/5] fix(clients): Update `log line too long` message (#611) Fixes https://github.com/cloudquery/cloudquery/issues/5665, by making this log message look less like an error. I could go with `Warn` too and maybe update the level of the other errors here (but won't do it unless someone finds those confusing). The current message can cause users to think that there is data loss or something wrong with the sync --- --- clients/destination/v0/destination.go | 6 +++--- clients/source/v0/source.go | 6 +++--- internal/logging/log_reader.go | 7 +++++-- internal/logging/log_reader_test.go | 2 +- 4 files changed, 12 insertions(+), 9 deletions(-) diff --git a/clients/destination/v0/destination.go b/clients/destination/v0/destination.go index cc3cf1f420..6310282b10 100644 --- a/clients/destination/v0/destination.go +++ b/clients/destination/v0/destination.go @@ -153,16 +153,16 @@ func (c *Client) newManagedClient(ctx context.Context, path string) error { break } if errors.Is(err, logging.ErrLogLineToLong) { - c.logger.Err(err).Str("line", string(line)).Msg("skipping too long log line") + c.logger.Info().Str("line", string(line)).Msg("truncated destination plugin log line") continue } if err != nil { - c.logger.Err(err).Msg("failed to read log line from plugin") + c.logger.Err(err).Msg("failed to read log line from destination plugin") break } var structuredLogLine map[string]any if err := json.Unmarshal(line, &structuredLogLine); err != nil { - c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from plugin") + c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from destination plugin") } else { logging.JSONToLog(c.logger, structuredLogLine) } diff --git a/clients/source/v0/source.go b/clients/source/v0/source.go index aed302db46..f0f4c367f1 100644 --- a/clients/source/v0/source.go +++ b/clients/source/v0/source.go @@ -150,16 +150,16 @@ func (c *Client) newManagedClient(ctx context.Context, path string) error { break } if errors.Is(err, logging.ErrLogLineToLong) { - c.logger.Err(err).Str("line", string(line)).Msg("skipping too long log line") + c.logger.Info().Str("line", string(line)).Msg("truncated source plugin log line") continue } if err != nil { - c.logger.Err(err).Msg("failed to read log line from plugin") + c.logger.Err(err).Msg("failed to read log line from source plugin") break } var structuredLogLine map[string]any if err := json.Unmarshal(line, &structuredLogLine); err != nil { - c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from plugin") + c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from source plugin") } else { logging.JSONToLog(c.logger, structuredLogLine) } diff --git a/internal/logging/log_reader.go b/internal/logging/log_reader.go index 269f74d15e..1af40bca8d 100644 --- a/internal/logging/log_reader.go +++ b/internal/logging/log_reader.go @@ -9,7 +9,10 @@ import ( // logReaderPrefixLen is used when returning a partial line as context in NextLine const logReaderPrefixLen = 1000 -var ErrLogLineToLong = errors.New("log line too long, discarding") +var ( + ErrLogLineToLong = errors.New("log line too long, discarding") + ellipsis = []byte("...") +) // logReader is a custom implementation similar to bufio.Scanner, but provides a way to handle lines // (or tokens) that exceed the buffer size. @@ -52,5 +55,5 @@ func (r *LogReader) NextLine() ([]byte, error) { return nil, err } } - return prefix, ErrLogLineToLong + return append(prefix, ellipsis...), ErrLogLineToLong } diff --git a/internal/logging/log_reader_test.go b/internal/logging/log_reader_test.go index 94343ae22a..3b4852b914 100644 --- a/internal/logging/log_reader_test.go +++ b/internal/logging/log_reader_test.go @@ -45,7 +45,7 @@ func Test_LogReader(t *testing.T) { name: "very long line", text: longStr(10000000), wantLines: []string{ - longStr(logReaderPrefixLen), + longStr(logReaderPrefixLen) + "...", }, wantErr: true, }, From 33ab32a690e99c00cf412097960a1d14efcff281 Mon Sep 17 00:00:00 2001 From: Yevgeny Pats Date: Mon, 23 Jan 2023 05:32:19 -0500 Subject: [PATCH 4/5] feat: Add version discovery service (#619) This adds discovery service to support multiple versions. Follow-up to this: https://github.com/cloudquery/plugin-sdk/pull/610 --- Makefile | 1 + clients/discovery/v0/constants.go | 5 + clients/discovery/v0/discovery.go | 203 ++++++++++++++ clients/discovery/v0/terminate.go | 30 ++ clients/discovery/v0/terminate_windows.go | 20 ++ clients/source/v1/source.go | 4 +- cloudquery/discovery/v0/discovery.proto | 16 ++ internal/pb/discovery/v0/discovery.pb.go | 261 ++++++++++++++++++ internal/pb/discovery/v0/discovery_grpc.pb.go | 107 +++++++ .../servers/destination/v0/destinations.go | 24 +- internal/servers/discovery/v0/discovery.go | 16 ++ serve/destination.go | 7 +- serve/source.go | 6 + 13 files changed, 685 insertions(+), 15 deletions(-) create mode 100644 clients/discovery/v0/constants.go create mode 100644 clients/discovery/v0/discovery.go create mode 100644 clients/discovery/v0/terminate.go create mode 100644 clients/discovery/v0/terminate_windows.go create mode 100644 cloudquery/discovery/v0/discovery.proto create mode 100644 internal/pb/discovery/v0/discovery.pb.go create mode 100644 internal/pb/discovery/v0/discovery_grpc.pb.go create mode 100644 internal/servers/discovery/v0/discovery.go diff --git a/Makefile b/Makefile index 5605f02a40..425f19118b 100644 --- a/Makefile +++ b/Makefile @@ -10,6 +10,7 @@ lint: gen-proto: protoc --proto_path=. --go_out . --go_opt=module="github.com/cloudquery/plugin-sdk" --go-grpc_out=. --go-grpc_opt=module="github.com/cloudquery/plugin-sdk" cloudquery/base/v0/base.proto cloudquery/destination/v0/destination.proto cloudquery/source/v0/source.proto protoc --proto_path=. --go_out . --go_opt=module="github.com/cloudquery/plugin-sdk" --go-grpc_out=. --go-grpc_opt=module="github.com/cloudquery/plugin-sdk" cloudquery/source/v1/source.proto + protoc --proto_path=. --go_out . --go_opt=module="github.com/cloudquery/plugin-sdk" --go-grpc_out=. --go-grpc_opt=module="github.com/cloudquery/plugin-sdk" cloudquery/discovery/v0/discovery.proto .PHONY: benchmark benchmark: diff --git a/clients/discovery/v0/constants.go b/clients/discovery/v0/constants.go new file mode 100644 index 0000000000..9651915154 --- /dev/null +++ b/clients/discovery/v0/constants.go @@ -0,0 +1,5 @@ +package discovery + +const ( + maxMsgSize = 100 * 1024 * 1024 // 100 MiB +) diff --git a/clients/discovery/v0/discovery.go b/clients/discovery/v0/discovery.go new file mode 100644 index 0000000000..c556b7afc0 --- /dev/null +++ b/clients/discovery/v0/discovery.go @@ -0,0 +1,203 @@ +package discovery + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "io" + "net" + "os" + "os/exec" + "path/filepath" + "strings" + "sync" + + "github.com/cloudquery/plugin-sdk/internal/logging" + pb "github.com/cloudquery/plugin-sdk/internal/pb/discovery/v0" + "github.com/cloudquery/plugin-sdk/internal/random" + "github.com/cloudquery/plugin-sdk/registry" + "github.com/cloudquery/plugin-sdk/specs" + "github.com/rs/zerolog" + "google.golang.org/grpc" + "google.golang.org/grpc/credentials/insecure" +) + +type Client struct { + pbClient pb.DiscoveryClient + directory string + cmd *exec.Cmd + logger zerolog.Logger + userConn *grpc.ClientConn + conn *grpc.ClientConn + grpcSocketName string + noSentry bool + wg *sync.WaitGroup +} + +type ClientOption func(*Client) + +func WithDirectory(directory string) func(*Client) { + return func(c *Client) { + c.directory = directory + } +} + +func WithGrpcConn(userConn *grpc.ClientConn) func(*Client) { + return func(c *Client) { + // we use a different variable here because we don't want to close a connection that wasn't created by us. + c.userConn = userConn + } +} + +func NewClient(ctx context.Context, registrySpec specs.Registry, pluginType registry.PluginType, path string, version string, opts ...ClientOption) (*Client, error) { + var err error + c := &Client{ + directory: registry.DefaultDownloadDir, + wg: &sync.WaitGroup{}, + } + for _, opt := range opts { + opt(c) + } + switch registrySpec { + case specs.RegistryGrpc: + if c.userConn == nil { + c.conn, err = grpc.DialContext(ctx, path, + grpc.WithTransportCredentials(insecure.NewCredentials()), + grpc.WithDefaultCallOptions( + grpc.MaxCallRecvMsgSize(maxMsgSize), + grpc.MaxCallSendMsgSize(maxMsgSize), + ), + ) + if err != nil { + return nil, fmt.Errorf("failed to dial grpc source plugin at %s: %w", path, err) + } + c.pbClient = pb.NewDiscoveryClient(c.conn) + } else { + c.pbClient = pb.NewDiscoveryClient(c.userConn) + } + return c, nil + case specs.RegistryLocal: + if err := c.newManagedClient(ctx, path); err != nil { + return nil, err + } + case specs.RegistryGithub: + pathSplit := strings.Split(path, "/") + if len(pathSplit) != 2 { + return nil, fmt.Errorf("invalid github plugin path: %s. format should be owner/repo", path) + } + org, name := pathSplit[0], pathSplit[1] + localPath := filepath.Join(c.directory, "plugins", string(pluginType), org, name, version, "plugin") + localPath = registry.WithBinarySuffix(localPath) + if err := registry.DownloadPluginFromGithub(ctx, localPath, org, name, version, pluginType); err != nil { + return nil, err + } + if err := c.newManagedClient(ctx, localPath); err != nil { + return nil, err + } + default: + return nil, fmt.Errorf("unsupported registry %s", registrySpec) + } + + return c, nil +} + +// newManagedClient starts a new discovery plugin process from local file, connects to it via gRPC server +// and returns a new Client +func (c *Client) newManagedClient(ctx context.Context, path string) error { + c.grpcSocketName = random.GenerateRandomUnixSocketName() + // spawn the plugin first and then connect + args := []string{"serve", "--network", "unix", "--address", c.grpcSocketName, + "--log-level", c.logger.GetLevel().String(), "--log-format", "json"} + if c.noSentry { + args = append(args, "--no-sentry") + } + cmd := exec.CommandContext(ctx, path, args...) + reader, err := cmd.StdoutPipe() + if err != nil { + return fmt.Errorf("failed to get stdout pipe: %w", err) + } + cmd.Stderr = os.Stderr + if err := cmd.Start(); err != nil { + return fmt.Errorf("failed to start destination plugin %s: %w", path, err) + } + + c.cmd = cmd + + c.wg.Add(1) + go func() { + defer c.wg.Done() + lr := logging.NewLogReader(reader) + for { + line, err := lr.NextLine() + if errors.Is(err, io.EOF) { + break + } + if errors.Is(err, logging.ErrLogLineToLong) { + c.logger.Info().Str("line", string(line)).Msg("truncated destination plugin log line") + continue + } + if err != nil { + c.logger.Err(err).Msg("failed to read log line from destination plugin") + break + } + var structuredLogLine map[string]any + if err := json.Unmarshal(line, &structuredLogLine); err != nil { + c.logger.Err(err).Str("line", string(line)).Msg("failed to unmarshal log line from destination plugin") + } else { + logging.JSONToLog(c.logger, structuredLogLine) + } + } + }() + + dialer := func(ctx context.Context, addr string) (net.Conn, error) { + d := &net.Dialer{} + return d.DialContext(ctx, "unix", addr) + } + c.conn, err = grpc.DialContext(ctx, c.grpcSocketName, grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithBlock(), grpc.WithContextDialer(dialer)) + if err != nil { + if err := cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill plugin process") + } + return err + } + c.pbClient = pb.NewDiscoveryClient(c.conn) + return nil +} + +func (c *Client) GetVersions(ctx context.Context) ([]string, error) { + res, err := c.pbClient.GetVersions(ctx, &pb.GetVersions_Request{}) + if err != nil { + return nil, fmt.Errorf("failed to call GetVersions: %w", err) + } + return res.Versions, nil +} + +// Terminate is used only in conjunction with NewManagedClient. +// It closes the connection it created, kills the spawned process and removes the socket file. +func (c *Client) Terminate() error { + // wait for log streaming to complete before returning from this function + defer c.wg.Wait() + + if c.grpcSocketName != "" { + defer func() { + if err := os.RemoveAll(c.grpcSocketName); err != nil { + c.logger.Error().Err(err).Msg("failed to remove destination socket file") + } + }() + } + + if c.conn != nil { + if err := c.conn.Close(); err != nil { + c.logger.Error().Err(err).Msg("failed to close gRPC connection to destination plugin") + } + c.conn = nil + } + if c.cmd != nil && c.cmd.Process != nil { + if err := c.terminateProcess(); err != nil { + return err + } + } + + return nil +} diff --git a/clients/discovery/v0/terminate.go b/clients/discovery/v0/terminate.go new file mode 100644 index 0000000000..1b11c4e103 --- /dev/null +++ b/clients/discovery/v0/terminate.go @@ -0,0 +1,30 @@ +//go:build !windows + +package discovery + +import ( + "fmt" + "os" + "time" +) + +func (c *Client) terminateProcess() error { + if err := c.cmd.Process.Signal(os.Interrupt); err != nil { + c.logger.Error().Err(err).Msg("failed to send interrupt signal to plugin") + } + timer := time.AfterFunc(5*time.Second, func() { + if err := c.cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill plugin") + } + }) + st, err := c.cmd.Process.Wait() + timer.Stop() + if err != nil { + return err + } + if !st.Success() { + return fmt.Errorf("plugin process exited with status %s", st.String()) + } + + return nil +} diff --git a/clients/discovery/v0/terminate_windows.go b/clients/discovery/v0/terminate_windows.go new file mode 100644 index 0000000000..387f6e35dc --- /dev/null +++ b/clients/discovery/v0/terminate_windows.go @@ -0,0 +1,20 @@ +//go:build windows + +package discovery + +func (c *Client) terminateProcess() error { + if err := c.cmd.Process.Kill(); err != nil { + c.logger.Error().Err(err).Msg("failed to kill plugin") + } + st, err := c.cmd.Process.Wait() + if err != nil { + return err + } + if !st.Success() { + // on windows there is no way to shutdown gracefully via signal. Maybe we can do it via grpc api? + // though it is a bit strange to expose api to shutdown a server :thinking?: + c.logger.Info().Msgf("plugin process exited with %s", st.String()) + } + + return nil +} diff --git a/clients/source/v1/source.go b/clients/source/v1/source.go index 0ab4aa71cb..a21823e37c 100644 --- a/clients/source/v1/source.go +++ b/clients/source/v1/source.go @@ -226,12 +226,12 @@ func (c *Client) GetTables(ctx context.Context) ([]*schema.Table, error) { return tables, nil } -func (c *Client) GetDynamicTables(ctx context.Context) ([]*schema.Table, error) { +func (c *Client) GetDynamicTables(ctx context.Context) (schema.Tables, error) { res, err := c.pbClient.GetDynamicTables(ctx, &pb.GetDynamicTables_Request{}) if err != nil { return nil, fmt.Errorf("failed to call GetDynamicTables: %w", err) } - var tables []*schema.Table + var tables schema.Tables if err := json.Unmarshal(res.Tables, &tables); err != nil { return nil, fmt.Errorf("failed to unmarshal tables: %w", err) } diff --git a/cloudquery/discovery/v0/discovery.proto b/cloudquery/discovery/v0/discovery.proto new file mode 100644 index 0000000000..8894df6c2a --- /dev/null +++ b/cloudquery/discovery/v0/discovery.proto @@ -0,0 +1,16 @@ +syntax = "proto3"; +package cloudquery.discovery.v0; + +option go_package = "github.com/cloudquery/plugin-sdk/internal/pb/discovery/v0;discovery"; + +service Discovery { + // Get the name of the plugin + rpc GetVersions(GetVersions.Request) returns (GetVersions.Response); +} + +message GetVersions { + message Request {} + message Response { + repeated string versions = 1; + } +} diff --git a/internal/pb/discovery/v0/discovery.pb.go b/internal/pb/discovery/v0/discovery.pb.go new file mode 100644 index 0000000000..16ab84a0c0 --- /dev/null +++ b/internal/pb/discovery/v0/discovery.pb.go @@ -0,0 +1,261 @@ +// Code generated by protoc-gen-go. DO NOT EDIT. +// versions: +// protoc-gen-go v1.28.1 +// protoc v3.21.12 +// source: cloudquery/discovery/v0/discovery.proto + +package discovery + +import ( + protoreflect "google.golang.org/protobuf/reflect/protoreflect" + protoimpl "google.golang.org/protobuf/runtime/protoimpl" + reflect "reflect" + sync "sync" +) + +const ( + // Verify that this generated code is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) + // Verify that runtime/protoimpl is sufficiently up-to-date. + _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) +) + +type GetVersions struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetVersions) Reset() { + *x = GetVersions{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[0] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersions) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersions) ProtoMessage() {} + +func (x *GetVersions) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[0] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersions.ProtoReflect.Descriptor instead. +func (*GetVersions) Descriptor() ([]byte, []int) { + return file_cloudquery_discovery_v0_discovery_proto_rawDescGZIP(), []int{0} +} + +type GetVersions_Request struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields +} + +func (x *GetVersions_Request) Reset() { + *x = GetVersions_Request{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[1] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersions_Request) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersions_Request) ProtoMessage() {} + +func (x *GetVersions_Request) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[1] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersions_Request.ProtoReflect.Descriptor instead. +func (*GetVersions_Request) Descriptor() ([]byte, []int) { + return file_cloudquery_discovery_v0_discovery_proto_rawDescGZIP(), []int{0, 0} +} + +type GetVersions_Response struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Versions []string `protobuf:"bytes,1,rep,name=versions,proto3" json:"versions,omitempty"` +} + +func (x *GetVersions_Response) Reset() { + *x = GetVersions_Response{} + if protoimpl.UnsafeEnabled { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[2] + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + ms.StoreMessageInfo(mi) + } +} + +func (x *GetVersions_Response) String() string { + return protoimpl.X.MessageStringOf(x) +} + +func (*GetVersions_Response) ProtoMessage() {} + +func (x *GetVersions_Response) ProtoReflect() protoreflect.Message { + mi := &file_cloudquery_discovery_v0_discovery_proto_msgTypes[2] + if protoimpl.UnsafeEnabled && x != nil { + ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) + if ms.LoadMessageInfo() == nil { + ms.StoreMessageInfo(mi) + } + return ms + } + return mi.MessageOf(x) +} + +// Deprecated: Use GetVersions_Response.ProtoReflect.Descriptor instead. +func (*GetVersions_Response) Descriptor() ([]byte, []int) { + return file_cloudquery_discovery_v0_discovery_proto_rawDescGZIP(), []int{0, 1} +} + +func (x *GetVersions_Response) GetVersions() []string { + if x != nil { + return x.Versions + } + return nil +} + +var File_cloudquery_discovery_v0_discovery_proto protoreflect.FileDescriptor + +var file_cloudquery_discovery_v0_discovery_proto_rawDesc = []byte{ + 0x0a, 0x27, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x64, 0x69, 0x73, + 0x63, 0x6f, 0x76, 0x65, 0x72, 0x79, 0x2f, 0x76, 0x30, 0x2f, 0x64, 0x69, 0x73, 0x63, 0x6f, 0x76, + 0x65, 0x72, 0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x17, 0x63, 0x6c, 0x6f, 0x75, 0x64, + 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x64, 0x69, 0x73, 0x63, 0x6f, 0x76, 0x65, 0x72, 0x79, 0x2e, + 0x76, 0x30, 0x22, 0x40, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, + 0x73, 0x1a, 0x09, 0x0a, 0x07, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x26, 0x0a, 0x08, + 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1a, 0x0a, 0x08, 0x76, 0x65, 0x72, 0x73, + 0x69, 0x6f, 0x6e, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x09, 0x52, 0x08, 0x76, 0x65, 0x72, 0x73, + 0x69, 0x6f, 0x6e, 0x73, 0x32, 0x77, 0x0a, 0x09, 0x44, 0x69, 0x73, 0x63, 0x6f, 0x76, 0x65, 0x72, + 0x79, 0x12, 0x6a, 0x0a, 0x0b, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x73, + 0x12, 0x2c, 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x64, 0x69, + 0x73, 0x63, 0x6f, 0x76, 0x65, 0x72, 0x79, 0x2e, 0x76, 0x30, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x65, + 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x2d, + 0x2e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2e, 0x64, 0x69, 0x73, 0x63, + 0x6f, 0x76, 0x65, 0x72, 0x79, 0x2e, 0x76, 0x30, 0x2e, 0x47, 0x65, 0x74, 0x56, 0x65, 0x72, 0x73, + 0x69, 0x6f, 0x6e, 0x73, 0x2e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x45, 0x5a, + 0x43, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x63, 0x6c, 0x6f, 0x75, + 0x64, 0x71, 0x75, 0x65, 0x72, 0x79, 0x2f, 0x70, 0x6c, 0x75, 0x67, 0x69, 0x6e, 0x2d, 0x73, 0x64, + 0x6b, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x70, 0x62, 0x2f, 0x64, 0x69, + 0x73, 0x63, 0x6f, 0x76, 0x65, 0x72, 0x79, 0x2f, 0x76, 0x30, 0x3b, 0x64, 0x69, 0x73, 0x63, 0x6f, + 0x76, 0x65, 0x72, 0x79, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, +} + +var ( + file_cloudquery_discovery_v0_discovery_proto_rawDescOnce sync.Once + file_cloudquery_discovery_v0_discovery_proto_rawDescData = file_cloudquery_discovery_v0_discovery_proto_rawDesc +) + +func file_cloudquery_discovery_v0_discovery_proto_rawDescGZIP() []byte { + file_cloudquery_discovery_v0_discovery_proto_rawDescOnce.Do(func() { + file_cloudquery_discovery_v0_discovery_proto_rawDescData = protoimpl.X.CompressGZIP(file_cloudquery_discovery_v0_discovery_proto_rawDescData) + }) + return file_cloudquery_discovery_v0_discovery_proto_rawDescData +} + +var file_cloudquery_discovery_v0_discovery_proto_msgTypes = make([]protoimpl.MessageInfo, 3) +var file_cloudquery_discovery_v0_discovery_proto_goTypes = []interface{}{ + (*GetVersions)(nil), // 0: cloudquery.discovery.v0.GetVersions + (*GetVersions_Request)(nil), // 1: cloudquery.discovery.v0.GetVersions.Request + (*GetVersions_Response)(nil), // 2: cloudquery.discovery.v0.GetVersions.Response +} +var file_cloudquery_discovery_v0_discovery_proto_depIdxs = []int32{ + 1, // 0: cloudquery.discovery.v0.Discovery.GetVersions:input_type -> cloudquery.discovery.v0.GetVersions.Request + 2, // 1: cloudquery.discovery.v0.Discovery.GetVersions:output_type -> cloudquery.discovery.v0.GetVersions.Response + 1, // [1:2] is the sub-list for method output_type + 0, // [0:1] is the sub-list for method input_type + 0, // [0:0] is the sub-list for extension type_name + 0, // [0:0] is the sub-list for extension extendee + 0, // [0:0] is the sub-list for field type_name +} + +func init() { file_cloudquery_discovery_v0_discovery_proto_init() } +func file_cloudquery_discovery_v0_discovery_proto_init() { + if File_cloudquery_discovery_v0_discovery_proto != nil { + return + } + if !protoimpl.UnsafeEnabled { + file_cloudquery_discovery_v0_discovery_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersions); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_discovery_v0_discovery_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersions_Request); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + file_cloudquery_discovery_v0_discovery_proto_msgTypes[2].Exporter = func(v interface{}, i int) interface{} { + switch v := v.(*GetVersions_Response); i { + case 0: + return &v.state + case 1: + return &v.sizeCache + case 2: + return &v.unknownFields + default: + return nil + } + } + } + type x struct{} + out := protoimpl.TypeBuilder{ + File: protoimpl.DescBuilder{ + GoPackagePath: reflect.TypeOf(x{}).PkgPath(), + RawDescriptor: file_cloudquery_discovery_v0_discovery_proto_rawDesc, + NumEnums: 0, + NumMessages: 3, + NumExtensions: 0, + NumServices: 1, + }, + GoTypes: file_cloudquery_discovery_v0_discovery_proto_goTypes, + DependencyIndexes: file_cloudquery_discovery_v0_discovery_proto_depIdxs, + MessageInfos: file_cloudquery_discovery_v0_discovery_proto_msgTypes, + }.Build() + File_cloudquery_discovery_v0_discovery_proto = out.File + file_cloudquery_discovery_v0_discovery_proto_rawDesc = nil + file_cloudquery_discovery_v0_discovery_proto_goTypes = nil + file_cloudquery_discovery_v0_discovery_proto_depIdxs = nil +} diff --git a/internal/pb/discovery/v0/discovery_grpc.pb.go b/internal/pb/discovery/v0/discovery_grpc.pb.go new file mode 100644 index 0000000000..bcdab222be --- /dev/null +++ b/internal/pb/discovery/v0/discovery_grpc.pb.go @@ -0,0 +1,107 @@ +// Code generated by protoc-gen-go-grpc. DO NOT EDIT. +// versions: +// - protoc-gen-go-grpc v1.2.0 +// - protoc v3.21.12 +// source: cloudquery/discovery/v0/discovery.proto + +package discovery + +import ( + context "context" + grpc "google.golang.org/grpc" + codes "google.golang.org/grpc/codes" + status "google.golang.org/grpc/status" +) + +// This is a compile-time assertion to ensure that this generated file +// is compatible with the grpc package it is being compiled against. +// Requires gRPC-Go v1.32.0 or later. +const _ = grpc.SupportPackageIsVersion7 + +// DiscoveryClient is the client API for Discovery service. +// +// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. +type DiscoveryClient interface { + // Get the name of the plugin + GetVersions(ctx context.Context, in *GetVersions_Request, opts ...grpc.CallOption) (*GetVersions_Response, error) +} + +type discoveryClient struct { + cc grpc.ClientConnInterface +} + +func NewDiscoveryClient(cc grpc.ClientConnInterface) DiscoveryClient { + return &discoveryClient{cc} +} + +func (c *discoveryClient) GetVersions(ctx context.Context, in *GetVersions_Request, opts ...grpc.CallOption) (*GetVersions_Response, error) { + out := new(GetVersions_Response) + err := c.cc.Invoke(ctx, "/cloudquery.discovery.v0.Discovery/GetVersions", in, out, opts...) + if err != nil { + return nil, err + } + return out, nil +} + +// DiscoveryServer is the server API for Discovery service. +// All implementations must embed UnimplementedDiscoveryServer +// for forward compatibility +type DiscoveryServer interface { + // Get the name of the plugin + GetVersions(context.Context, *GetVersions_Request) (*GetVersions_Response, error) + mustEmbedUnimplementedDiscoveryServer() +} + +// UnimplementedDiscoveryServer must be embedded to have forward compatible implementations. +type UnimplementedDiscoveryServer struct { +} + +func (UnimplementedDiscoveryServer) GetVersions(context.Context, *GetVersions_Request) (*GetVersions_Response, error) { + return nil, status.Errorf(codes.Unimplemented, "method GetVersions not implemented") +} +func (UnimplementedDiscoveryServer) mustEmbedUnimplementedDiscoveryServer() {} + +// UnsafeDiscoveryServer may be embedded to opt out of forward compatibility for this service. +// Use of this interface is not recommended, as added methods to DiscoveryServer will +// result in compilation errors. +type UnsafeDiscoveryServer interface { + mustEmbedUnimplementedDiscoveryServer() +} + +func RegisterDiscoveryServer(s grpc.ServiceRegistrar, srv DiscoveryServer) { + s.RegisterService(&Discovery_ServiceDesc, srv) +} + +func _Discovery_GetVersions_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { + in := new(GetVersions_Request) + if err := dec(in); err != nil { + return nil, err + } + if interceptor == nil { + return srv.(DiscoveryServer).GetVersions(ctx, in) + } + info := &grpc.UnaryServerInfo{ + Server: srv, + FullMethod: "/cloudquery.discovery.v0.Discovery/GetVersions", + } + handler := func(ctx context.Context, req interface{}) (interface{}, error) { + return srv.(DiscoveryServer).GetVersions(ctx, req.(*GetVersions_Request)) + } + return interceptor(ctx, in, info, handler) +} + +// Discovery_ServiceDesc is the grpc.ServiceDesc for Discovery service. +// It's only intended for direct use with grpc.RegisterService, +// and not to be introspected or modified (even as a copy) +var Discovery_ServiceDesc = grpc.ServiceDesc{ + ServiceName: "cloudquery.discovery.v0.Discovery", + HandlerType: (*DiscoveryServer)(nil), + Methods: []grpc.MethodDesc{ + { + MethodName: "GetVersions", + Handler: _Discovery_GetVersions_Handler, + }, + }, + Streams: []grpc.StreamDesc{}, + Metadata: "cloudquery/discovery/v0/discovery.proto", +} diff --git a/internal/servers/destination/v0/destinations.go b/internal/servers/destination/v0/destinations.go index fa78a91dfb..c6a2c0ff70 100644 --- a/internal/servers/destination/v0/destinations.go +++ b/internal/servers/destination/v0/destinations.go @@ -1,4 +1,4 @@ -package servers +package destination import ( "context" @@ -17,19 +17,19 @@ import ( "google.golang.org/grpc/status" ) -type DestinationServer struct { +type Server struct { pb.UnimplementedDestinationServer Plugin *destination.Plugin Logger zerolog.Logger } -func (*DestinationServer) GetProtocolVersion(context.Context, *pbBase.GetProtocolVersion_Request) (*pbBase.GetProtocolVersion_Response, error) { +func (*Server) GetProtocolVersion(context.Context, *pbBase.GetProtocolVersion_Request) (*pbBase.GetProtocolVersion_Response, error) { return &pbBase.GetProtocolVersion_Response{ Version: 2, }, nil } -func (s *DestinationServer) Configure(ctx context.Context, req *pbBase.Configure_Request) (*pbBase.Configure_Response, error) { +func (s *Server) Configure(ctx context.Context, req *pbBase.Configure_Request) (*pbBase.Configure_Response, error) { var spec specs.Destination if err := json.Unmarshal(req.Config, &spec); err != nil { return nil, status.Errorf(codes.InvalidArgument, "failed to unmarshal spec: %v", err) @@ -37,19 +37,19 @@ func (s *DestinationServer) Configure(ctx context.Context, req *pbBase.Configure return &pbBase.Configure_Response{}, s.Plugin.Init(ctx, s.Logger, spec) } -func (s *DestinationServer) GetName(context.Context, *pbBase.GetName_Request) (*pbBase.GetName_Response, error) { +func (s *Server) GetName(context.Context, *pbBase.GetName_Request) (*pbBase.GetName_Response, error) { return &pbBase.GetName_Response{ Name: s.Plugin.Name(), }, nil } -func (s *DestinationServer) GetVersion(context.Context, *pbBase.GetVersion_Request) (*pbBase.GetVersion_Response, error) { +func (s *Server) GetVersion(context.Context, *pbBase.GetVersion_Request) (*pbBase.GetVersion_Response, error) { return &pbBase.GetVersion_Response{ Version: s.Plugin.Version(), }, nil } -func (s *DestinationServer) Migrate(ctx context.Context, req *pb.Migrate_Request) (*pb.Migrate_Response, error) { +func (s *Server) Migrate(ctx context.Context, req *pb.Migrate_Request) (*pb.Migrate_Response, error) { var tables []*schema.Table if err := json.Unmarshal(req.Tables, &tables); err != nil { return nil, status.Errorf(codes.InvalidArgument, "failed to unmarshal tables: %v", err) @@ -57,13 +57,13 @@ func (s *DestinationServer) Migrate(ctx context.Context, req *pb.Migrate_Request return &pb.Migrate_Response{}, s.Plugin.Migrate(ctx, tables) } -func (*DestinationServer) Write(pb.Destination_WriteServer) error { +func (*Server) Write(pb.Destination_WriteServer) error { return status.Errorf(codes.Unimplemented, "method Write is deprecated please upgrade client") } // Note the order of operations in this method is important! // Trying to insert into the `resources` channel before starting the reader goroutine will cause a deadlock. -func (s *DestinationServer) Write2(msg pb.Destination_Write2Server) error { +func (s *Server) Write2(msg pb.Destination_Write2Server) error { resources := make(chan schema.DestinationResource) r, err := msg.Recv() @@ -131,7 +131,7 @@ func (s *DestinationServer) Write2(msg pb.Destination_Write2Server) error { } } -func (s *DestinationServer) GetMetrics(context.Context, *pb.GetDestinationMetrics_Request) (*pb.GetDestinationMetrics_Response, error) { +func (s *Server) GetMetrics(context.Context, *pb.GetDestinationMetrics_Request) (*pb.GetDestinationMetrics_Response, error) { stats := s.Plugin.Metrics() b, err := json.Marshal(stats) if err != nil { @@ -142,7 +142,7 @@ func (s *DestinationServer) GetMetrics(context.Context, *pb.GetDestinationMetric }, nil } -func (s *DestinationServer) DeleteStale(ctx context.Context, req *pb.DeleteStale_Request) (*pb.DeleteStale_Response, error) { +func (s *Server) DeleteStale(ctx context.Context, req *pb.DeleteStale_Request) (*pb.DeleteStale_Response, error) { var tables schema.Tables if err := json.Unmarshal(req.Tables, &tables); err != nil { return nil, status.Errorf(codes.InvalidArgument, "failed to unmarshal tables: %v", err) @@ -154,6 +154,6 @@ func (s *DestinationServer) DeleteStale(ctx context.Context, req *pb.DeleteStale return &pb.DeleteStale_Response{}, nil } -func (s *DestinationServer) Close(ctx context.Context, _ *pb.Close_Request) (*pb.Close_Response, error) { +func (s *Server) Close(ctx context.Context, _ *pb.Close_Request) (*pb.Close_Response, error) { return &pb.Close_Response{}, s.Plugin.Close(ctx) } diff --git a/internal/servers/discovery/v0/discovery.go b/internal/servers/discovery/v0/discovery.go new file mode 100644 index 0000000000..d02a563135 --- /dev/null +++ b/internal/servers/discovery/v0/discovery.go @@ -0,0 +1,16 @@ +package discovery + +import ( + "context" + + pb "github.com/cloudquery/plugin-sdk/internal/pb/discovery/v0" +) + +type Server struct { + pb.UnimplementedDiscoveryServer + Versions []string +} + +func (s *Server) GetVersions(context.Context, *pb.GetVersions_Request) (*pb.GetVersions_Response, error) { + return &pb.GetVersions_Response{Versions: s.Versions}, nil +} diff --git a/serve/destination.go b/serve/destination.go index 2489ca457d..401c0c574f 100644 --- a/serve/destination.go +++ b/serve/destination.go @@ -9,7 +9,9 @@ import ( "sync" pbv0 "github.com/cloudquery/plugin-sdk/internal/pb/destination/v0" + pbdiscoveryv0 "github.com/cloudquery/plugin-sdk/internal/pb/discovery/v0" servers "github.com/cloudquery/plugin-sdk/internal/servers/destination/v0" + discoveryServerV0 "github.com/cloudquery/plugin-sdk/internal/servers/discovery/v0" "github.com/cloudquery/plugin-sdk/plugins/destination" "github.com/getsentry/sentry-go" grpczerolog "github.com/grpc-ecosystem/go-grpc-middleware/providers/zerolog/v2" @@ -108,10 +110,13 @@ func newCmdDestinationServe(serve *destinationServe) *cobra.Command { grpc.MaxRecvMsgSize(pbv0.MaxMsgSize), grpc.MaxSendMsgSize(pbv0.MaxMsgSize), ) - pbv0.RegisterDestinationServer(s, &servers.DestinationServer{ + pbv0.RegisterDestinationServer(s, &servers.Server{ Plugin: serve.plugin, Logger: logger, }) + pbdiscoveryv0.RegisterDiscoveryServer(s, &discoveryServerV0.Server{ + Versions: []string{"v0"}, + }) version := serve.plugin.Version() if serve.sentryDSN != "" && !strings.EqualFold(version, "development") && !noSentry { diff --git a/serve/source.go b/serve/source.go index 13cfca9aa4..6c01f776a0 100644 --- a/serve/source.go +++ b/serve/source.go @@ -8,8 +8,10 @@ import ( "strings" "sync" + pbdiscoveryv0 "github.com/cloudquery/plugin-sdk/internal/pb/discovery/v0" pbv0 "github.com/cloudquery/plugin-sdk/internal/pb/source/v0" pbv1 "github.com/cloudquery/plugin-sdk/internal/pb/source/v1" + discoveryServerV0 "github.com/cloudquery/plugin-sdk/internal/servers/discovery/v0" serversv0 "github.com/cloudquery/plugin-sdk/internal/servers/source/v0" serversv1 "github.com/cloudquery/plugin-sdk/internal/servers/source/v1" "github.com/cloudquery/plugin-sdk/plugins/source" @@ -125,6 +127,10 @@ func newCmdSourceServe(serve *sourceServe) *cobra.Command { Plugin: serve.plugin, Logger: logger, }) + pbdiscoveryv0.RegisterDiscoveryServer(s, &discoveryServerV0.Server{ + Versions: []string{"v0", "v1"}, + }) + version := serve.plugin.Version() if serve.sentryDSN != "" && !strings.EqualFold(version, "development") && !noSentry { From 79d7217d19cecba457d7996b9fd81d6d0c5a532c Mon Sep 17 00:00:00 2001 From: CloudQuery Bot <102256036+cq-bot@users.noreply.github.com> Date: Mon, 23 Jan 2023 12:40:22 +0200 Subject: [PATCH 5/5] chore(main): Release v1.28.0 (#615) :robot: I have created a release *beep* *boop* --- ## [1.28.0](https://github.com/cloudquery/plugin-sdk/compare/v1.27.0...v1.28.0) (2023-01-23) ### Features * Add version discovery service ([#619](https://github.com/cloudquery/plugin-sdk/issues/619)) ([33ab32a](https://github.com/cloudquery/plugin-sdk/commit/33ab32a690e99c00cf412097960a1d14efcff281)) * Dynamic tables and introduce proto versioning ([#610](https://github.com/cloudquery/plugin-sdk/issues/610)) ([448232c](https://github.com/cloudquery/plugin-sdk/commit/448232c8789350c8fb071902d33a5c5f07d2b82c)) ### Bug Fixes * **clients:** Update `log line too long` message ([#611](https://github.com/cloudquery/plugin-sdk/issues/611)) ([0d3ff48](https://github.com/cloudquery/plugin-sdk/commit/0d3ff48d4a8ce324b5685c3df9196943d09b2eba)) * Simplify client naming conventions ([#617](https://github.com/cloudquery/plugin-sdk/issues/617)) ([38b136b](https://github.com/cloudquery/plugin-sdk/commit/38b136b9aa15dc049f9b66dcd4ceca60fa7bdca6)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --- CHANGELOG.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 036928be9f..f18f492866 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,20 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.28.0](https://github.com/cloudquery/plugin-sdk/compare/v1.27.0...v1.28.0) (2023-01-23) + + +### Features + +* Add version discovery service ([#619](https://github.com/cloudquery/plugin-sdk/issues/619)) ([33ab32a](https://github.com/cloudquery/plugin-sdk/commit/33ab32a690e99c00cf412097960a1d14efcff281)) +* Dynamic tables and introduce proto versioning ([#610](https://github.com/cloudquery/plugin-sdk/issues/610)) ([448232c](https://github.com/cloudquery/plugin-sdk/commit/448232c8789350c8fb071902d33a5c5f07d2b82c)) + + +### Bug Fixes + +* **clients:** Update `log line too long` message ([#611](https://github.com/cloudquery/plugin-sdk/issues/611)) ([0d3ff48](https://github.com/cloudquery/plugin-sdk/commit/0d3ff48d4a8ce324b5685c3df9196943d09b2eba)) +* Simplify client naming conventions ([#617](https://github.com/cloudquery/plugin-sdk/issues/617)) ([38b136b](https://github.com/cloudquery/plugin-sdk/commit/38b136b9aa15dc049f9b66dcd4ceca60fa7bdca6)) + ## [1.27.0](https://github.com/cloudquery/plugin-sdk/compare/v1.26.0...v1.27.0) (2023-01-17)