Scheduling Compute Engine with startup scripts
Hi people! Today we will create a compute engine instance that boots and shuts down at specific times with a custom scripts. Let me explain a little bit more.
Google Compute Engine (GCE) is a Infrastructure as a Service (IaaS)that allows developers to run their job on physical hardware. They offer so many types of virtual machines. You can modify RAMs, CPUs, auxiliary storages and even operating systems on them.
Ok, but how can we schedule them? I mean we are not going to hire someone to push the button every time, for sure. Right? Or are we?
That’s where Google Cloud Scheduler, Cloud Functions and Cloud Pub/Sub come in. It seems lot stuff but trust me it’s not. With help of Cloud Scheduler, we can set specific times and payloads(i will explain that later). Cloud Scheduler triggers the Pub/Sub. Pub/Sub gets the payload and triggers the Cloud Functions. And finally Cloud Functions boots up or shuts down our instance with a startup script.
So let’s get started. We are going to use website to setup this. You can use gcloud command-line tool if you want. First of all you should enable billing to access Compute Engine. However Google give you 300 USD$ credit for one year!
Okay let’s go to Compute Engine Instances and create a new project. Then click Create Instance at the top. Set the name dev-instance . Select region us-west1 and for zone select us-west1-b . Go to Management, security, disks, networking, sole tenancy. Under management click Add label and set env for key, dev for value. Then click create. Congratulations, you have your virtual machine up and running. But we are not done yet. We will setup Cloud Functions with Pub/Sub.
Let’s head over and go to Cloud Functions. Click Create Function. Set your function’s name startInstancePubSub. For trigger, select Cloud Pub/Sub. For Topic, select Create new topic…. A new dialog box will appear. In this box, set Name start-instance-event. Click create. Dialog box will disappear. For Runtime, select Node.js 8. Select index.js and replace the code with this.
const Compute = require('@google-cloud/compute');
const compute = new Compute();
exports.startInstancePubSub = async (event, context, callback) => {
try {
const payload = _validatePayload(
JSON.parse(Buffer.from(event.data, 'base64').toString())
);
const options = {filter: `labels.${payload.label}`};
const [vms] = await compute.getVMs(options);
await Promise.all(
vms.map(async instance => {
if (payload.zone === instance.zone.id) {
const [operation] = await compute
.zone(payload.zone)
.vm(instance.name)
.start();
// Operation pending
return operation.promise();
}
})
);
const message = `Successfully started instance(s)`;
console.log(message);
callback(null, message);
} catch (err) {
console.log(err);
callback(err);
}
};const _validatePayload = payload => {
if (!payload.zone) {
throw new Error(`Attribute 'zone' missing from payload`);
} else if (!payload.label) {
throw new Error(`Attribute 'label' missing from payload`);
}
return payload;
};
Then select package.json. And replace the code with this.
{
"name": "cloud-functions-schedule-instance",
"version": "0.1.0",
"private": true,
"license": "Apache-2.0",
"author": "Google Inc.",
"repository": {
"type": "git",
"url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
},
"engines": {
"node": ">=8.0.0"
},
"scripts": {
"test": "mocha test/*.test.js --timeout=20000"
},
"devDependencies": {
"@google-cloud/nodejs-repo-tools": "^3.3.0",
"mocha": "^6.0.0",
"proxyquire": "^2.0.0",
"sinon": "^7.0.0"
},
"dependencies": {
"@google-cloud/compute": "^1.0.0"
}
}For Function to execute, set startInstancePubSub. Then click create. That was our starter function. Let’s setup stopper function.
Again, click Create Function. Name it stopInstancePubSub. For Trigger, select Cloud Pub/Sub. For Topic, select Create new topic…. In the dialog box, set name stop-instance-event an click create. For Runtime, select node.js 8 . Select index.js tab and replace the code with this.
const Compute = require('@google-cloud/compute');
const compute = new Compute();
exports.stopInstancePubSub = async (event, context, callback) => {
try {
const payload = _validatePayload(
JSON.parse(Buffer.from(event.data, 'base64').toString())
);
const options = {filter: `labels.${payload.label}`};
const [vms] = await compute.getVMs(options);
await Promise.all(
vms.map(async instance => {
if (payload.zone === instance.zone.id) {
const [operation] = await compute
.zone(payload.zone)
.vm(instance.name)
.stop();
// Operation pending
return operation.promise();
} else {
return Promise.resolve();
}
})
); const message = `Successfully stopped instance(s)`;
console.log(message);
callback(null, message);
} catch (err) {
console.log(err);
callback(err);
}
};
const _validatePayload = payload => {
if (!payload.zone) {
throw new Error(`Attribute 'zone' missing from payload`);
} else if (!payload.label) {
throw new Error(`Attribute 'label' missing from payload`);
}
return payload;
};
Then select package.json and replace the code with this.
{
"name": "cloud-functions-schedule-instance",
"version": "0.1.0",
"private": true,
"license": "Apache-2.0",
"author": "Google Inc.",
"repository": {
"type": "git",
"url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
},
"engines": {
"node": ">=8.0.0"
},
"scripts": {
"test": "mocha test/*.test.js --timeout=20000"
},
"devDependencies": {
"@google-cloud/nodejs-repo-tools": "^3.3.0",
"mocha": "^6.0.0",
"proxyquire": "^2.0.0",
"sinon": "^7.0.0"
},
"dependencies": {
"@google-cloud/compute": "^1.0.0"
}
}Okay we setted up our Functions and Pub/Sub. Next, we will setup Scheduler. Go to Cloud Scheduler. Click Create Job. Name it startup-dev-instances . For frequency, you can enter whatever cron time you want. For example, * * * * * means every minute. Or 0 9 * * 1–5 means at 09:00 on every day from monday to friday. For Timezone, select United States/Los Angeles . For Target, select Pub/Sub . For Topic, enter start-instance-event . For Payload, (as i mentioned earlier) enter this:
{"zone":"us-west1-b","label":"env=dev"}Then click Create. We setted up our starter scheduler, let’s setup stopper one.
Click Create Job. Name it shutdown-dev-instances . For frequency, enter your frequency. For Timezone, select United States/Los Angeles . For Target, select Pub/Sub . For Topic, enter stop-instance-event . For Payload, (as i mentioned earlier) enter this:
{"zone":"us-west1-b","label":"env=dev"}Then click Create . Congratulations! You scheduled your instance. But we haven’t assign any startup script. Let’s do that.
Go to Compute Engine. Click on dev-instance. Click Edit at the top of page. Find Custom metadata. Click Add item. For key, enter startup-script. For value, you can enter your script. Once you assign a startup script, everytime the vm boots up, it will execute the startup script. Let’s do a python script example.
#! /bin/bash
python -<< EOF # start of python script
print(‘Hello World!’)
EOF # end of python script
This script will be executed and print “Hello World!” everytime VM boots up. Let’s check it. Go to Compute Engine. Select dev-instance . Click Start. When it boots up, click SSH. It will open a window. This is your bash command line. Type this code to see logs:
tail -n 100 /var/log/syslog
And we are done. You can execute your own script. If you have any more question feel free to ask me. Have a nice day :)
