jeffreyaven · Mar 15, 2022 · Mar 15, 2022
diff --git a/README.md b/README.md
@@ -9,7 +9,7 @@
 
 # Deploy, Manage and Query Cloud Infrastructure using SQL
 
-[Documentation](https://docs.stackql.io/)]  [Developer Guide](/docs/developer_guide.md)]
+[[Documentation](https://docs.stackql.io/)]  [[Developer Guide](/docs/developer_guide.md)] [[BYO Providers](/docs/registry_contribution.md)]
 
 ## Cloud infrastructure coding using SQL
 
@@ -21,6 +21,14 @@
 
 ----
 
+## Provider development
+
+Keen to expose some new functionality though `stackql`?  We are very keen on this!  
+
+Please see [registry_contribution.md](/docs/registry_contribution.md).
+
+---
+
 ## Design
 
 [HLDD](/docs/high-level-design.md)

diff --git a/build/.gitignore b/build/.gitignore
@@ -1,2 +1,3 @@
 *
-!.gitignore
+!.gitignore
+!build.md
diff --git a/build/build.md b/build/build.md
@@ -0,0 +1,4 @@
+
+## Build Directory
+
+This directory is the location for building and testing artifacts.  Various walkthroughs and tests depend upon the executable being pre-built to this directory.
diff --git a/docs/developer_guide.md b/docs/developer_guide.md
@@ -1,6 +1,26 @@
 
 # StackQL Developer Guide
 
+## Provider development
+
+Keen to expose some new functionality though `stackql`?  We are very keen on this!  
+
+Please see [registry_contribution.md](/docs/registry_contribution.md).
+
+## Provider Authentication
+
+At this stage, authentication config must be specified for each provider, even for unauthorized ones.  Supported auth types are:
+
+- `api_key`.
+- `basic`.
+- `interactive` for interactive oAuth, thus far only google supported via `gcloud` command line tool.
+- `service_account` for json style private keys (eg: google service accounts).
+- `null_auth` for unauthenticated providers.
+
+If you want further auth types or discover bugs, please raise an issue.
+
+Examples are present [here](/docs/examples.md).
+
 ## Concurrency considerations
 
 In server mode, a thread pool issues one thread to handle each connection.

diff --git a/docs/registry_contribution.md b/docs/registry_contribution.md
@@ -0,0 +1,78 @@
+
+# Provider Registry Contribution
+
+Community development of provider functionality is key to `stackql` and our mission.
+Here, we describe the process to develop, test and integrate a new provider from zero to General Availability.
+
+We include an example integration for a simple Provider, `publicapis`.
+
+## Workflow
+
+1. Create a Provider definition document.
+2. Create at least one Service definition document.
+3. Run `stackql` with config to support local Provider development.
+4. Iterate and verify that `stackql` works as expected against your new provider.
+5. Submit a Pull Request against the Provider Registry repository.
+
+
+### 1. Create a Provider definition document
+
+This is a yaml or json document which contains Provider metadata and reference(s) to any Service document(s) through which it will expose functionality.
+
+Example as per [examples/registry/src/publicapis/v1/provider.yaml](/examples/registry/src/publicapis/v1/provider.yaml).
+
+### 2. Create at least one Service definition document.
+
+This is an [openapi document spec](https://swagger.io/specification/) in either yaml or json format, **plus** a legal annotation `components.x-stackQL-resources`, which defines the Resource portion of the `stackql` heirarchy.
+
+Example as per [examples/registry/src/publicapis/v1/services/api-v1.yaml](/examples/registry/src/publicapis/v1/services/api-v1.yaml).
+
+
+### 3. Run `stackql` with config to support local Provider development.
+
+Configure `stackql` to use your docs, bia the `--registry` command line argument.  The [example integration section](#Configuring-StackQL-to-consume-a-local-development-registry) expands upon this.
+
+### 4. Iterate and verify that `stackql` works as expected against your new provider.
+
+Iterate upon steps 2 and 3 until API coverage and functionality fulfil your requirements.
+
+### 5. Submit a Pull Request against the Provider Registry repository.
+
+As per [the Provider Registry Contributor Guide](https://github.com/stackql/stackql-provider-registry/blob/main/.github/CONTRIBUTING.md).  The team will review as rapidly as possible and work with you to complete the integration.  Once this is done, your functionality with be available via `registry pull...`
+
+## Example integration
+
+In this walkthrough, we demonstrate the bare bones of integrating a simple Provider,
+`publicapis`, which is included here within source control.
+
+The assumptions for this walkthrough are:
+
+1. You have either built (eg: via `cmake` as per [the readme](/README.md#build)) or copied the `stackql` executable for your platform into the [build](/build) directory.
+2. The current working directory is same as this document, otherwise relative paths to `pwd` will need to be adjusted.
+
+### Configuring StackQL to consume a local development registry
+
+Registry access config:
+
+```bash
+PROVIDER_REGISTRY_ROOT_DIR="$(pwd)/../examples/registry"
+
+REG_STR='{ "url": "file://'${PROVIDER_REGISTRY_ROOT_DIR}'", "localDocRoot": "'${PROVIDER_REGISTRY_ROOT_DIR}'",  "verifyConfig": {"nopVerify": true } }'
+```
+
+Provider auth config as per [developer guide](/docs/developer_guide.md#provider-authentication):
+
+```bash
+AUTH_STR='{ "publicapis": { "type": "null_auth" }  }'
+```
+
+### Interacting with the local development registry
+
+```bash
+$(pwd)/../build/stackql --auth="${AUTH_STR}" --registry="${REG_STR}" exec "select API from publicapis.api.apis where API like 'Dog%' limit 10;"
+
+$(pwd)/../build/stackql --auth="${AUTH_STR}" --registry="${REG_STR}" exec "select API from publicapis.api.random where title =  'Dog';"
+
+## The single, anonmyous column returned from selecing an array of strings is a core issue to fix separate to this
+$(pwd)/../build/stackql --auth="${AUTH_STR}" --registry="${REG_STR}" exec "select * from publicapis.api.categories limit 5;"
+```
diff --git a/examples/README.md b/examples/README.md
diff --git a/internal/stackql/cmd/root.go b/internal/stackql/cmd/root.go
@@ -46,7 +46,7 @@ var (
 )
 
 const (
-	defaultRegistryUrlString string = "https://cdn.statically.io/gh/stackql/stackql-provider-registry/intial-devel/providers"
+	defaultRegistryUrlString string = "https://cdn.statically.io/gh/stackql/stackql-provider-registry/main/providers"
 )
 
 var SemVersion string = fmt.Sprintf("%s.%s.%s", BuildMajorVersion, BuildMinorVersion, BuildPatchVersion)
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,4 @@

		## Build Directory

		This directory is the location for building and testing artifacts. Various walkthroughs and tests depend upon the executable being pre-built to this directory.
-Original file line number
+Diff line change
@@ Expand Up / @@ -46,7 +46,7 @@ var ( @@
     )
     const (
-    	defaultRegistryUrlString string = "https://cdn.statically.io/gh/stackql/stackql-provider-registry/intial-devel/providers"
+    	defaultRegistryUrlString string = "https://cdn.statically.io/gh/stackql/stackql-provider-registry/main/providers"
     )
     var SemVersion string = fmt.Sprintf("%s.%s.%s", BuildMajorVersion, BuildMinorVersion, BuildPatchVersion)
@@ Expand Down @@