We are using grafana, influxdb & telegraf for server monitoring. Everything going great we are getting email alerts also if something goes wrong thanks to Grafana Alerting Engine. Recently we missed a couple of server alerts when we discussed with teams we realized most of the people don’t see emails often or sometimes emails go to spam or go to some Gmail filter. Everyone said if we get SMS instead of emails we will never miss.
But the problem is how we will integrate our SMS provider API with Grafana Alerting Engine. Finally, the twist came in this almost feel good story 😉.
Without twists, life will get bored. After little googling, we understood that through Grafana webhook notification we can do it. Another problem is Grafana webhook notification has fixed format so we can’t directly send this to our SMS provider. Grafana webhook notification sends information in below format about a state change over HTTP to a custom endpoint
{
"title": "My alert",
"ruleId": 1,
"ruleName": "Load peaking!",
"ruleUrl": "http://url.to.grafana/db/dashboard/my_dashboard?panelId=2",
"state": "alerting",
"imageUrl": "http://s3.image.url",
"message": "Load is peaking. Make sure the traffic is real and spin up more webfronts",
"evalMatches": [
{
"metric": "requests",
"tags": {},
"value": 122
}
]
}
So we need to write a middle layer or proxy for receiving grafana webhook alert notification and transform the data into our SMS gateway provider format. Writing a small application for this doesn’t make sense. We need a server also to deploy this and it hardly gets 1 or 2 requests daily. Then I thought why can’t we use AWS lambda. Which is dead cheap and I don’t need to maintain another application.
So, I wrote a small AWS lambda function to get started which I want to share with you guys. I hope you have AWS account if not please create. Let’s
start.
After logging in AWS account type lambda in AWS services search. You will see below screen. Then click on Create a function
Then I chose Author from scratch. Then give name, choose Runtime(I
chose Node.js 8.10), Then choose Role(I chose Create new role from template(s), Give Role name, choose Policy templates — I chose Basic Edge Lambda permissions).
Then Click on Create Function. Now your lambda function created. Let’s write the code.
Go below you will see Function code section write your code there. I wrote this function.
exports.handler = (event, context, callback) => { console.log("Inside my lambda function"); callback(null, "Hello World"); };
Now we want to test what output will get. Let’s test this. We can do it by clicking on Test button. Then it will show a popup, leave all options as it is, give an Event name and click on Create button.
Now click on Test button again. You will get Execution result.
Congrats!!! your lambda function almost ready.
Now let’s make it an API. To run lambda function we need a trigger point for that choose API Gateway. Click on API Gateway in Add triggers section.
Now let’s configure our trigger. Go to Configure triggers section and choose API (I chose Create a new API), give some API name, give a Deployment stage, choose Security(I chose Open). That’s it, Click on Add.
Then click on Save. We are almost there. Expand API Details then you will find URL to which we need to hit.
Don’t use this URL yet we need to do one more thing. If you open now you will get {“message”: “Internal server error”} because your lambda function not giving in the format your API gateway proxy expecting.
Need to do this code change to make it work.
exports.handler = (event, context, callback) => { console.log("Inside my lambda function"); let response = { statusCode: 200, body: "Hello World" }; callback(null, response); };
We need to publish our latest lambda function.
Click on Save. Then click on Actions. Then click on Publish new version.
You will get another popup for Version description.
Congrats!!!. Your first serverless API ready.
Let’s say you want to send some parameters to your API and want to use in your code. For that need to do one step. Go to Services and type API Gateway. Now you will see your API.
Now click on your API and then click on ANY of your API resource method.
This screen is actually execution flow of our API. Now click on Method Request.
Now I want to send one get parameter and JSON object in Request Body. Now expand URL Query String Parameters and then click on Add query
string. Give a parameter name and click on right button. Then Expand Request Body click on Add model then give Content type(I gave application/json) then choose Model name(I chose Empty)and click on right button.
Now let’s use those in our lambda script.
Go to Lambda service again. Open your lambda and do the code changes.
exports.handler = (event, context, callback) => { console.log("Inside my lambda function"); let response = { statusCode: 200, body: JSON.stringify({"parameter": event.queryStringParameters.parameter, "requestBody": event.body}) }; callback(null, response); };
Publish it again. Let’s hit our API again with get parameter and request body.
Peace. Happy Coding.