Skip to content

enhanced-robot-testing-changeover #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
general-kroll-4-life merged 1 commit into from
Apr 1, 2022
Merged

enhanced-robot-testing-changeover #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions .vscode/launch.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,17 +87,15 @@
},
// select name from google.compute.disks where project = 'lab-kr-network-01' and zone = 'australia-southeast1-a';
{
"name": "DEBUG select compute.disks ... ;",
"name": "DEBUG select compute.acceleratorTypes ... ;",
"type": "go",
"request": "launch",
"mode": "debug",
"program": "${workspaceFolder}/stackql",
"args": [
"exec",
"--auth={ \"google\": { \"credentialsfilepath\": \"${workspaceFolder}/keys/sa-key.json\" }, \"okta\": { \"credentialsfilepath\": \"${workspaceFolder}/keys/okta-token.txt\", \"type\": \"api_key\" } }",
"select name from google.compute.disks where project = 'lab-kr-network-01' and zone = 'australia-southeast1-a';",
"-o",
"json"
"select kind, name from google.compute.acceleratorTypes where project = 'lab-kr-network-01' and zone = 'australia-southeast1-a' order by name desc;"
],
},
// show extended resources from okta.group
Expand Down
11 changes: 11 additions & 0 deletions test/assets/expected/simple-select/compute-accelerator-type/select-zone-list-desc.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
|-------------------------|---------------------|
| kind | name |
|-------------------------|---------------------|
| compute#acceleratorType | nvidia-tesla-t4-vws |
|-------------------------|---------------------|
| compute#acceleratorType | nvidia-tesla-t4 |
|-------------------------|---------------------|
| compute#acceleratorType | nvidia-tesla-p4-vws |
|-------------------------|---------------------|
| compute#acceleratorType | nvidia-tesla-p4 |
|-------------------------|---------------------|
24 changes: 20 additions & 4 deletions test/functional/stackql_mocked_from_cmd_line.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,36 @@ Suite Teardown Terminate All Processes
*** Test Cases ***
Google Container Agg Desc
Should StackQL Exec Equal
... ${REGISTRY_NO_VERIFY_CFG_STR}
... ${SELECT_CONTAINER_SUBNET_AGG_DESC}
... ${SELECT_CONTAINER_SUBNET_AGG_DESC_EXPECTED}

Google Container Agg Asc
Should StackQL Exec Equal
... ${REGISTRY_NO_VERIFY_CFG_STR}
... ${SELECT_CONTAINER_SUBNET_AGG_ASC}
... ${SELECT_CONTAINER_SUBNET_AGG_ASC_EXPECTED}

Google IAM Policy Agg
Should StackQL Exec Equal
... ${REGISTRY_NO_VERIFY_CFG_STR}
... \-\-infile\=${GET_IAM_POLICY_AGG_ASC_INPUT_FILE}
... ${GET_IAM_POLICY_AGG_ASC_EXPECTED}
... \-o\=csv


Google AcceleratorTypes SQL verb pre changeover
Should StackQL Exec Equal
... ${REGISTRY_NO_VERIFY_CFG_STR}
... ${SELECT_ACCELERATOR_TYPES_DESC}
... ${SELECT_ACCELERATOR_TYPES_DESC_EXPECTED}

Google AcceleratorTypes SQL verb post changeover
Should StackQL Exec Equal
... ${REGISTRY_SQL_VERB_CONTRIVED_NO_VERIFY_CFG_STR}
... ${SELECT_ACCELERATOR_TYPES_DESC}
... ${SELECT_ACCELERATOR_TYPES_DESC_EXPECTED}

Basic Query mTLS Returns OK
Should PG Client Inline Contain
... ${PSQL_MTLS_CONN_STR}
Expand Down Expand Up @@ -62,11 +78,11 @@ Prepare StackQL Environment


Run StackQL Exec Command
[Arguments] ${_EXEC_CMD_STR} @{varargs}
[Arguments] ${_REGISTRY_CFG_STR} ${_EXEC_CMD_STR} @{varargs}
Set Environment Variable OKTA_SECRET_KEY ${OKTA_SECRET_STR}
${result} = Run Process
... ${STACKQL_EXE}
... exec \-\-registry\=${REGISTRY_NO_VERIFY_CFG_STR}
... exec \-\-registry\=${_REGISTRY_CFG_STR}
... \-\-auth\=${AUTH_CFG_STR}
... \-\-tls.allowInsecure\=true
... ${_EXEC_CMD_STR} @{varargs}
Expand All @@ -76,8 +92,8 @@ Run StackQL Exec Command


Should StackQL Exec Equal
[Arguments] ${_EXEC_CMD_STR} ${_EXEC_CMD_EXPECTED_OUTPUT} @{varargs}
${result} = Run StackQL Exec Command ${_EXEC_CMD_STR} @{varargs}
[Arguments] ${_REGISTRY_CFG_STR} ${_EXEC_CMD_STR} ${_EXEC_CMD_EXPECTED_OUTPUT} @{varargs}
${result} = Run StackQL Exec Command ${_REGISTRY_CFG_STR} ${_EXEC_CMD_STR} @{varargs}
Should Be Equal ${result.stdout} ${_EXEC_CMD_EXPECTED_OUTPUT}

Start StackQL PG Server mTLS
Expand Down
13 changes: 13 additions & 0 deletions test/functional/variables/stackql_context.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@ def get_unix_path(pathStr :str) -> str:
"nopVerify": True
}
}
_REGISTRY_SQL_VERB_CONTRIVED_NO_VERIFY_CFG = {
"url": f"file://{get_unix_path(REGISTRY_ROOT_CANONICAL)}",
"localDocRoot": f"{get_unix_path(REGISTRY_ROOT_CANONICAL)}",
"srcPrefix": "registry-verb-matching-src",
"useEmbedded": False,
"verifyConfig": {
"nopVerify": True
}
}
_REGISTRY_CANONICAL_CFG = {
"url": f"file://{get_unix_path(REGISTRY_ROOT_CANONICAL)}",
"localDocRoot": f"{get_unix_path(REGISTRY_ROOT_CANONICAL)}",
Expand Down Expand Up @@ -79,6 +88,7 @@ def get_unix_path(pathStr :str) -> str:
OKTA_SECRET_STR = f.read()

REGISTRY_NO_VERIFY_CFG_STR = json.dumps(_REGISTRY_NO_VERIFY_CFG)
REGISTRY_SQL_VERB_CONTRIVED_NO_VERIFY_CFG_STR = json.dumps(_REGISTRY_SQL_VERB_CONTRIVED_NO_VERIFY_CFG)
REGISTRY_CANONICAL_CFG_STR = json.dumps(_REGISTRY_CANONICAL_CFG)
REGISTRY_DEPRECATED_CFG_STR = json.dumps(_REGISTRY_DEPRECATED_CFG)
AUTH_CFG_STR = json.dumps(_AUTH_CFG)
Expand All @@ -104,11 +114,14 @@ def get_unix_path(pathStr :str) -> str:

SELECT_CONTAINER_SUBNET_AGG_DESC = "select ipCidrRange, sum(5) cc from google.container.`projects.aggregated.usableSubnetworks` where projectsId = 'testing-project' group by \"ipCidrRange\" having sum(5) >= 5 order by ipCidrRange desc;"
SELECT_CONTAINER_SUBNET_AGG_ASC = "select ipCidrRange, sum(5) cc from google.container.`projects.aggregated.usableSubnetworks` where projectsId = 'testing-project' group by \"ipCidrRange\" having sum(5) >= 5 order by ipCidrRange asc;"
SELECT_ACCELERATOR_TYPES_DESC = "select kind, name from google.compute.acceleratorTypes where project = 'testing-project' and zone = 'australia-southeast1-a' order by name desc;"

SELECT_CONTAINER_SUBNET_AGG_DESC_EXPECTED = get_output_from_local_file(os.path.join('test', 'assets', 'expected', 'aggregated-select', 'google', 'container', 'agg-subnetworks-allowed', 'table', 'simple-count-grouped-variant-desc.txt'))

SELECT_CONTAINER_SUBNET_AGG_ASC_EXPECTED = get_output_from_local_file(os.path.join('test', 'assets', 'expected', 'aggregated-select', 'google', 'container', 'agg-subnetworks-allowed', 'table', 'simple-count-grouped-variant-asc.txt'))

SELECT_ACCELERATOR_TYPES_DESC_EXPECTED = get_output_from_local_file(os.path.join('test', 'assets', 'expected', 'simple-select', 'compute-accelerator-type', 'select-zone-list-desc.txt'))

GET_IAM_POLICY_AGG_ASC_INPUT_FILE = os.path.join(REPOSITORY_ROOT, 'test', 'assets', 'input', 'select-exec-dependent-org-iam-policy.iql')

GET_IAM_POLICY_AGG_ASC_EXPECTED = get_output_from_local_file(os.path.join('test', 'assets', 'expected', 'aggregated-select', 'google', 'cloudresourcemanager', 'select-exec-getiampolicy-agg.csv'))
2 changes: 1 addition & 1 deletion test/mockserver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ mvn org.apache.maven.plugins:maven-dependency-plugin:3.2.0:copy -Dartifact=org.m
To run:

```bash
java -Dfile.encoding=UTF-8 -Dmockserver.initializationJsonPath=${HOME}/stackql/stackql-devel/test/server/expectations/static-expectations.json -jar /usr/local/lib/mockserver/mockserver-netty-jar-with-dependencies.jar -serverPort 1080 -logLevel INFO
java -Dfile.encoding=UTF-8 -Dmockserver.initializationJsonPath=${HOME}/stackql/stackql-devel/test/mockserver/expectations/static-gcp-expectations.json -jar /usr/local/lib/mockserver/mockserver-netty-jar-with-dependencies.jar -serverPort 1080 -logLevel INFO
```

### Expectations from local file
Expand Down
55 changes: 55 additions & 0 deletions test/mockserver/expectations/static-gcp-expectations.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,61 @@
}
}
},
{
"httpRequest": {
"method": "GET",
"path": "/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes"
},
"httpResponse": {
"body": {
"kind": "compute#acceleratorTypeList",
"id": "projects/testing-project/zones/australia-southeast1-a/acceleratorTypes",
"items": [
{
"kind": "compute#acceleratorType",
"id": "10010",
"creationTimestamp": "1969-12-31T16:00:00.000-08:00",
"name": "nvidia-tesla-p4",
"description": "NVIDIA Tesla P4",
"zone": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a",
"selfLink": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes/nvidia-tesla-p4",
"maximumCardsPerInstance": 4
},
{
"kind": "compute#acceleratorType",
"id": "10012",
"creationTimestamp": "1969-12-31T16:00:00.000-08:00",
"name": "nvidia-tesla-p4-vws",
"description": "NVIDIA Tesla P4 Virtual Workstation",
"zone": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a",
"selfLink": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes/nvidia-tesla-p4-vws",
"maximumCardsPerInstance": 4
},
{
"kind": "compute#acceleratorType",
"id": "10019",
"creationTimestamp": "1969-12-31T16:00:00.000-08:00",
"name": "nvidia-tesla-t4",
"description": "NVIDIA Tesla T4",
"zone": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a",
"selfLink": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes/nvidia-tesla-t4",
"maximumCardsPerInstance": 4
},
{
"kind": "compute#acceleratorType",
"id": "10020",
"creationTimestamp": "1969-12-31T16:00:00.000-08:00",
"name": "nvidia-tesla-t4-vws",
"description": "NVIDIA Tesla T4 Virtual Workstation",
"zone": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a",
"selfLink": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes/nvidia-tesla-t4-vws",
"maximumCardsPerInstance": 4
}
],
"selfLink": "https://www.googleapis.com/compute/beta/projects/testing-project/zones/australia-southeast1-a/acceleratorTypes"
}
}
},
{
"httpRequest": {
"path": "/v3/organizations/123456789012:getIamPolicy"
Expand Down
Loading