Skip to content

Commit 963efc6

Browse files
author
Ace Nassri
authored
[WIP] Remove emulator + shims from testing samples (GoogleCloudPlatform#1255)
* Remove emulator from testing samples + remove custom shim * Move HTTP integration test to functions framework * Address comments + clarify delays * Remove timeouts + shorten region-tagged blocks * Remove debug var * More region-tag-shortening * Use retries instead of delays * Add supertest back * Fix lint + use handleLinuxErrors helper func * Add delay package back + remove shim test * Use Node 8 syntax (for FF compatibility) * Update non-integration tests w/ Node 8 syntax + removed emulator * Switch to delay package in system tests * Fix lint * Specify BASE_URL for non-Emulator tests * Remove BASE_URL from non-system tests * Revamp system tests to get them passing * Move BASE_URL to Kokoro cfg + remove emulator script block * Add FF dependency * Switch to Node 8 image * Fix tests: deconflict PORT values * Fix tests: deconflict PORT values, take 2 * Tweak test values + remove dbg code * DEBUG: glinux is VERY angry, so debug in CI instead * Attempt 1: get stdout on Linux machines * Attempt 2: remove dbg code + use err object * Apply previous fix to all tests * Apply previous fixes to all samples * Increase FF timeouts * Switch from Node timeouts to Unix-based ones * Remove Linux failures handler, since Linux tolerates bash timeouts * Label system tests + increase timeout * Change bucket names to avoid conflicts * Add system test instructions to README * Fix lint * functions/pubsub: use separate port + Bash timeouts * Kokoro: revert FUNCTIONS_BUCKET change + use FUNCTIONS_DELETABLE_BUCKET instead * Typo-fix in pubsub sample * functions/pubsub: fix typos * FF: remove global dep + add to pubsub package.json * Fix bad pubsub FF URL * Reduce timeout for system tests * Fix package.json typo + increase functions/pubsub log backtrack period * DEBUG: debug pubsub system test * DBG: add more pubsub debugging * Revert debug commits
1 parent 33bdcf5 commit 963efc6

21 files changed

Lines changed: 637 additions & 794 deletions

‎.kokoro/build.sh‎

Lines changed: 2 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ export SENDGRID_API_KEY=$(cat $KOKORO_GFILE_DIR/secrets-sendgrid-api-key.txt)
4444
# Configure GCF variables
4545
export FUNCTIONS_TOPIC=integration-tests-instance
4646
export FUNCTIONS_BUCKET=$GCLOUD_PROJECT
47+
export FUNCTIONS_DELETABLE_BUCKET=$GCLOUD_PROJECT-functions
4748

4849
# functions/speech-to-speech
4950
export OUTPUT_BUCKET=$FUNCTIONS_BUCKET
@@ -59,7 +60,7 @@ export RESULT_TOPIC=$FUNCTIONS_TOPIC
5960
export RESULT_BUCKET=$FUNCTIONS_BUCKET
6061

6162
# functions/imagemagick
62-
export BLURRED_BUCKET_NAME=$GCLOUD_PROJECT-functions
63+
export BLURRED_BUCKET_NAME=$GCLOUD_PROJECT-imagick
6364

6465
# Configure IoT variables
6566
export NODEJS_IOT_EC_PUBLIC_KEY=${KOKORO_GFILE_DIR}/ec_public.pem
@@ -80,19 +81,6 @@ export GOOGLE_APPLICATION_CREDENTIALS=${KOKORO_GFILE_DIR}/secrets-key.json
8081
gcloud auth activate-service-account --key-file "$GOOGLE_APPLICATION_CREDENTIALS"
8182
gcloud config set project $GCLOUD_PROJECT
8283

83-
npm install -g @google-cloud/functions-framework
84-
85-
# Start functions emulator, if appropriate
86-
if [[ $PROJECT == functions/* ]] && grep --quiet functions-emulator package.json; then
87-
export BASE_URL="http://localhost:8010/${GCLOUD_PROJECT}/${GCF_REGION}"
88-
89-
export FUNCTIONS_LOG_PATH=$(pwd)/logs/cloud-functions-emulator.log
90-
npm install -g @google-cloud/functions-emulator
91-
touch "$FUNCTIONS_LOG_PATH"
92-
functions config set logFile "$FUNCTIONS_LOG_PATH"
93-
functions-emulator start
94-
fi
95-
9684
npm test
9785

9886
exit $?

‎.kokoro/functions/common.cfg‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,5 +12,11 @@ build_file: "nodejs-docs-samples/.kokoro/trampoline.sh"
1212
# Configure the docker image for kokoro-trampoline.
1313
env_vars: {
1414
key: "TRAMPOLINE_IMAGE"
15-
value: "gcr.io/cloud-docs-samples/nodejs"
15+
value: "gcr.io/cloud-devrel-kokoro-resources/node:8-user"
16+
}
17+
18+
# Configure the base URL of the function hosting platform
19+
env_vars: {
20+
key: "BASE_URL"
21+
value: "http://localhost:8080"
1622
}

‎.kokoro/functions/helloworld.cfg‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,10 @@ env_vars: {
1010
env_vars: {
1111
key: "TRAMPOLINE_BUILD_FILE"
1212
value: "github/nodejs-docs-samples/.kokoro/build.sh"
13-
}
13+
}
14+
15+
# Configure the base URL of the function hosting platform
16+
env_vars: {
17+
key: "BASE_URL"
18+
value: "http://us-central1-nodejs-docs-samples-tests.cloudfunctions.net"
19+
}

‎functions/helloworld/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,14 @@ See:
1414

1515
See the [Cloud Functions Hello World tutorial][tutorial].
1616

17+
**Note:** in order for the tests to run properly, you'll have to deploy some of the sample functions:
18+
19+
```
20+
gcloud functions deploy helloHttp --runtime nodejs8 --trigger-http
21+
gcloud functions deploy helloPubSub --trigger-topic $FUNCTIONS_TOPIC --runtime nodejs8
22+
gcloud functions deploy helloGCS --runtime nodejs8 --trigger-resource $FUNCTIONS_DELETABLE_BUCKET --trigger-event providers/cloud.storage/eventTypes/object.change
23+
```
24+
1725
## Run the tests
1826

1927
1. Read and follow the [prerequisites](../../../../#prerequisites).
@@ -31,7 +39,7 @@ See the [Cloud Functions Hello World tutorial][tutorial].
3139

3240
export GCF_REGION=us-central1
3341
export FUNCTIONS_TOPIC=[YOUR_PUBSUB_TOPIC]
34-
export FUNCTIONS_BUCKET=[YOUR_CLOUD_STORAGE_BUCKET]
42+
export FUNCTIONS_DELETABLE_BUCKET=[YOUR_CLOUD_STORAGE_BUCKET] # will be deleted by tests!
3543

3644
1. Run the tests:
3745

‎functions/helloworld/index.js‎

Lines changed: 20 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -57,8 +57,8 @@ exports.helloHttp = (req, res) => {
5757
* @param {object} event The Cloud Functions event.
5858
* @param {function} callback The callback function.
5959
*/
60-
exports.helloBackground = (event, callback) => {
61-
callback(null, `Hello ${event.data.name || 'World'}!`);
60+
exports.helloBackground = (data, context, callback) => {
61+
callback(null, `Hello ${data.name || 'World'}!`);
6262
};
6363
// [END functions_helloworld_background]
6464

@@ -68,42 +68,37 @@ exports.helloBackground = (event, callback) => {
6868
* This function is exported by index.js, and executed when
6969
* the trigger topic receives a message.
7070
*
71-
* @param {object} event The Cloud Functions event.
72-
* @param {function} callback The callback function.
71+
* @param {object} data The event payload.
72+
* @param {object} context The event metadata.
7373
*/
74-
exports.helloPubSub = (event, callback) => {
75-
const pubsubMessage = event.data;
76-
const name = pubsubMessage.data
77-
? Buffer.from(pubsubMessage.data, 'base64').toString()
74+
exports.helloPubSub = (data, context) => {
75+
const pubSubMessage = data;
76+
const name = pubSubMessage.data
77+
? Buffer.from(pubSubMessage.data, 'base64').toString()
7878
: 'World';
7979

8080
console.log(`Hello, ${name}!`);
81-
82-
callback();
8381
};
8482
// [END functions_helloworld_pubsub]
8583

8684
// [START functions_helloworld_storage]
8785
/**
8886
* Background Cloud Function to be triggered by Cloud Storage.
8987
*
90-
* @param {object} event The Cloud Functions event.
91-
* @param {function} callback The callback function.
88+
* @param {object} data The event payload.
89+
* @param {object} context The event metadata.
9290
*/
93-
exports.helloGCS = (event, callback) => {
94-
const file = event.data;
95-
91+
exports.helloGCS = (data, context) => {
92+
const file = data;
9693
if (file.resourceState === 'not_exists') {
9794
console.log(`File ${file.name} deleted.`);
9895
} else if (file.metageneration === '1') {
9996
// metageneration attribute is updated on metadata changes.
100-
// value is 1 if file was newly created or overwritten
97+
// on create value is 1
10198
console.log(`File ${file.name} uploaded.`);
10299
} else {
103100
console.log(`File ${file.name} metadata updated.`);
104101
}
105-
106-
callback();
107102
};
108103
// [END functions_helloworld_storage]
109104

@@ -114,11 +109,11 @@ exports.helloGCS = (event, callback) => {
114109
* @param {object} event The Cloud Functions event.
115110
* @param {function} callback The callback function.
116111
*/
117-
exports.helloGCSGeneric = (event, callback) => {
118-
const file = event.data;
112+
exports.helloGCSGeneric = (data, context, callback) => {
113+
const file = data;
119114

120-
console.log(` Event: ${event.eventId}`);
121-
console.log(` Event Type: ${event.eventType}`);
115+
console.log(` Event: ${context.eventId}`);
116+
console.log(` Event Type: ${context.eventType}`);
122117
console.log(` Bucket: ${file.bucket}`);
123118
console.log(` File: ${file.name}`);
124119
console.log(` Metageneration: ${file.metageneration}`);
@@ -136,7 +131,7 @@ exports.helloGCSGeneric = (event, callback) => {
136131
* @param {function} callback The callback function.
137132
*/
138133

139-
exports.helloError = (event, callback) => {
134+
exports.helloError = (data, context, callback) => {
140135
// [START functions_helloworld_error]
141136
// These WILL be reported to Stackdriver Error Reporting
142137
console.error(new Error('I failed you'));
@@ -154,7 +149,7 @@ exports.helloError = (event, callback) => {
154149
*/
155150
/* eslint-disable no-throw-literal */
156151

157-
exports.helloError2 = (event, callback) => {
152+
exports.helloError2 = (data, context, callback) => {
158153
// [START functions_helloworld_error]
159154
// These will NOT be reported to Stackdriver Error Reporting
160155
console.info(new Error('I failed you')); // Logging an Error object at the info level
@@ -171,7 +166,7 @@ exports.helloError2 = (event, callback) => {
171166
* @param {function} callback The callback function.
172167
*/
173168
/* eslint-disable */
174-
exports.helloError3 = (event, callback) => {
169+
exports.helloError3 = (data, context, callback) => {
175170
// This will NOT be reported to Stackdriver Error Reporting
176171
// [START functions_helloworld_error]
177172
callback('I failed you');

‎functions/helloworld/package.json‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,24 +12,31 @@
1212
"node": ">=8.0.0"
1313
},
1414
"scripts": {
15-
"e2e-test": "export FUNCTIONS_CMD='gcloud functions' && sh test/updateFunctions.sh && BASE_URL=\"https://$GCP_REGION-$GCLOUD_PROJECT.cloudfunctions.net/\" mocha test/*.test.js --timeout=60000 --exit",
16-
"test": "export FUNCTIONS_CMD='functions-emulator' && sh test/updateFunctions.sh && export BASE_URL=\"http://localhost:8010/$GCLOUD_PROJECT/$GCF_REGION\" && mocha test/*.test.js --timeout=60000 --exit",
17-
"system-test": "export FUNCTIONS_CMD='functions-emulator' && sh test/updateFunctions.sh && export BASE_URL=\"http://localhost:8010/$GCLOUD_PROJECT/$GCF_REGION\" && mocha test/*.test.js --timeout=60000 --exit"
15+
"unit-test": "mocha test/index.test.js test/*unit*test.js test/*integration*test.js --timeout=2000 --exit",
16+
"system-test": "mocha test/*system*test.js --timeout=60000 --exit",
17+
"test": "npm run unit-test && npm run system-test"
1818
},
1919
"dependencies": {
2020
"@google-cloud/debug-agent": "^4.0.0",
2121
"escape-html": "^1.0.3",
22-
"pug": "^2.0.3"
22+
"pug": "^2.0.3",
23+
"supertest": "^4.0.2"
2324
},
2425
"devDependencies": {
26+
"@google-cloud/functions-framework": "^1.1.1",
2527
"@google-cloud/nodejs-repo-tools": "^3.3.0",
26-
"@google-cloud/pubsub": "^0.28.0",
28+
"@google-cloud/pubsub": "^0.30.0",
2729
"@google-cloud/storage": "^3.0.0",
28-
"mocha": "^6.0.0",
30+
"child-process-promise": "^2.2.1",
31+
"delay": "^4.3.0",
2932
"express": "^4.16.3",
33+
"mocha": "^6.1.4",
34+
"moment": "^2.24.0",
35+
"promise-retry": "^1.1.1",
3036
"proxyquire": "^2.1.0",
37+
"request": "^2.88.0",
38+
"requestretry": "^4.0.0",
3139
"sinon": "^7.0.0",
32-
"supertest": "^4.0.0",
3340
"uuid": "^3.1.0",
3441
"yargs": "^14.0.0"
3542
},

‎functions/helloworld/shim.js‎

Lines changed: 0 additions & 153 deletions
This file was deleted.

0 commit comments

Comments
 (0)