Asynchronous Processing with AWS API Gateway and SNS
- Aws
- AWS API GATEWAY
- AWS SNS
- PARALLEL PROCESSING
When you want to build an asynchronous endpoint that can receive your request and process the task(s) in the background, AWS services can be used to achieve that. The technique I cover here allows for parallel asynchronous processing not just asynchronous. The two important services we need are API Gateway and SNS.
AWS API Gateway
AWS API Gateway can act as a proxy for your backend endpoints or lambda function or many AWS Services including SNS. Remember the maximum execution time cannot exceed 29 seconds.
SNS
SNS means a Simple Notification Service that can receive the message from the publisher and trigger the subscribers. The subscribers can include HTTP endpoint, Lambda, SMS or Email.
Configuration
Now you can create a new endpoint on API Gateway as below and map it to an SNS so that whenever you hit the endpoint it will publish a message to SNS, which in turn calls your backend API.
Step 1: Create a Topic under SNS
Step 2: Create an IAM Role to grant access to SNS from API Gateway
Step 3: Create an API endpoint as shown below,
See the config below,
AWS Service: SNS
Action: Publish
Execution Role: [You need to create an IAM Role with access to SNS Topic]
As you could see, I have used Query parameters with static values (included inside a single quote), alternatively, you could pass that from your request to the SNS using param mapping.
The Subject, Message, and TopicArn are the three values to pass. TopicArn can be found on the SNS Topic page.
Deploy your API and every time you do a GET request on this it will publish a message into the selected SNS Topic.
Now you could configure SNS subscribers that can do the job, for eg, configure Lambda function/HTTP Endpoint to run a logic. You could also see that this method allows for fanout, that is you could have “N number of parallel subscribers to be executed (say 10 lambda functions, each doing a different job and an Email that notifies the end user), this allows for parallel asynchronous processing.