How to automate copying an EC2 AMI from one region to another
31 May 2017
DevOps Automation
12 October 2017
Published by
BluePi
Data-Driven Business Transformation
piStats AWS Lambda Architecture
Problem Statement
piStats enables its clients to analyze the users on their site and send Push Notifications to engage and retain them for a longer duration. For this purpose the clients’ register their users’ on piStats using piStats SDKs.
The user base registered on piStats is unevenly distributed across different clients and their properties and the registration load is unpredictable – it is usually high for either first release with piStats integration or on a day when a special event occurs like elections.
To handle the unpredictable load piStats needs to scale up or down as the need occurs.
Solution and Flow
To handle the load of sudden spurt of registration and subscription and to remove the overhead of scaling up and down to hand in accordance with the load piStats utilizes AWS Lambda architecture. AWS Lambda is an event driven service from Amazon.
Whenever a new client starts using piStats and a release rolls out after integration, piStats observes a sudden surge of events. With this surge of event, AWS Lambda architecture scales up easily and handles all events without us having to intervene and removes the overhead of handling the load manually.
When the load reduces AWS Lambda architecture scales down itself and reduces the number of servers that had started to handle the sudden spurt of events.
Components:
- Kong
- AWS Lambda
- Redis
- Amazon RDS
- Amazon Kinesis
Flow
Any request hitting piStats land onto the nginx servers with Amazon ELB in front with autoscaling to handle unpredictable load of events. The nginx servers then forward the registration/subscription requests to AWS Lambda. This is where all the interaction with Redis, Amazon RDS and FCM(Firebase Cloud Messaging) occurs. After processing, these requests again go to the Nginx servers which in turn sends these requests to Amazon Kinesis for further processing by our storm servers deployed on Amazon EC2 machines. The storm servers perform the required enrichment and send the data for storage in our analytics database.
Problem Statement
piStats enables its clients to analyze the users on their site and send Push Notifications to engage and retain them for a longer duration. For this purpose the clients’ register their users’ on piStats using piStats SDKs.
The user base registered on piStats is unevenly distributed across different clients and their properties and the registration load is unpredictable – it is usually high for either first release with piStats integration or on a day when a special event occurs like elections.
To handle the unpredictable load piStats needs to scale up or down as the need occurs.
Solution and Flow
To handle the load of sudden spurt of registration and subscription and to remove the overhead of scaling up and down to hand in accordance with the load piStats utilizes AWS Lambda architecture. AWS Lambda is an event driven service from Amazon.
Whenever a new client starts using piStats and a release rolls out after integration, piStats observes a sudden surge of events. With this surge of event, AWS Lambda architecture scales up easily and handles all events without us having to intervene and removes the overhead of handling the load manually.
When the load reduces AWS Lambda architecture scales down itself and reduces the number of servers that had started to handle the sudden spurt of events.
Components:
- Kong
- AWS Lambda
- Redis
- Amazon RDS
- Amazon Kinesis
Flow
Any request hitting piStats land onto the nginx servers with Amazon ELB in front with autoscaling to handle unpredictable load of events. The nginx servers then forward the registration/subscription requests to AWS Lambda. This is where all the interaction with Redis, Amazon RDS and FCM(Firebase Cloud Messaging) occurs.
After processing, these requests again go to the Nginx servers which in turn sends these requests to Amazon Kinesis for further processing by our storm servers deployed on Amazon EC2 machines.
The storm servers perform the required enrichment and send the data for storage in our analytics database.
Contact Us
RELATED BLOGS
