Serverless Diary: Innovative Way to Multithreading in the cloud
I. Introduction
“The essence of abstraction is preserving information that is relevant in a given context, and forgetting information that is irrelevant in that context. — John V. Guttag”
The focus of the software architecture evolution has been on creating the next layer of abstraction. It allows us to focus our efforts on new unsolved challenges and not worry about re-inventing the wheel for complex resolved problems of the past. For example, looking back at the evolution of programming languages, first, we had machine languages, followed by procedural languages, object-oriented languages, and service-oriented architecture. This software evolution demonstrates abstraction layers built on top of the previous one, freeing programmers of low-level details like writing code for memory management, garbage collection, etc. With the public cloud era we are in, these abstraction levels have evolved further. All major public cloud providers offer FAAS or fully managed services like AWS lambda, AZURE functions, which hugely allows us to focus on just the code required for business logic without worrying about provisioning servers, managing capacity, scaling, etc.
The focus of this blog is to share an effective unorthodox alternative to multithreading.
II. The Challenge
Let me share one of my experiences around a design challenge that required x-number of tasks to be completed in near-real-time (completionTime<30 secs). Here,
x = variable that may increase or decrease based upon dynamically changing demand.
task = Unit Of Work (UOW)as defined by Unit of work pattern,
completionTime = total time in seconds for completion of x tasks.
So a traditional solution would be to write code that manages multithreading to execute these tasks in parallel. An example in java:
class CustomTask implements Runnable {public void run(){
try {
task(); // unit of work
}
catch (Exception e) {
System.out.println("Exception is caught");
}}}class Multithread {public static void main(String[] args){
int x = 50; // Number of threadsfor (int i = 0; i < x; i++) {
Thread object = new Thread(new CustomTask());
object.start();
}}}
Since this challenge is part of serverless series, Let us understand a few of the limitations of the above code when wrapped as lambda and deployed.
1. Multithreaded code within the lambda function usually results in hyperthreading rather than multithreading. For the example above, the code snippet deployed as lambda with 512MB of memory will result in 50 hyper threads (not multi-thread) on a single Core. A lambda can use up to 6 vCPU’s (before December 2020 this was limited to 1–2 cores), but then we are talking about having very fat lambdas (10GB memory), which isn’t an efficient, cost-effective, and scalable approach.
2. To add further complexity to the mix, multithreading on lambda is a bit different and you can’t just “fire and forget” a background task in Lambda, like you can with a normal Java application. You have to use Thread.join() or some other coordination mechanism to ensure that the background task finishes before the handler function returns. If you don’t, that task may never execute: there’s no guarantee that the Lambda container that was running it will ever be invoked again.
I want my execution time of all tasks to be less than 30 seconds regardless of the increase in demand. So what is a serverless approach for the given challenge?
III. The Solution
Let’s move up an abstraction level and solve this challenge via design rather than multithreading code. The serverless design presented below is a slight modification of the fanout pattern, but more custom to our use case.
Steps
1. Lambda with producer hat on, applies divide and conquer approach with each UOW published as a message(you can also opt for creating a bundle of messages with several UOW grouped as a message) on event stream like SQS (or kinesis data stream for more fine-grained control over the number of parallel consumers).
2. The Producer lambda is also registered as a listener to the SQS.
3. The number of lambdas (consumer) auto-scales based upon the queue depth of SQS and performs the tasks in parallel. Depending upon requirements you can exercise more control over scalability and performance by using the provisioned capacity of lambda functions.
IV. The Result & Learnings
The outcome of the above design approach using fully managed AWS services was a highly effective and scalable pattern performing hundreds of tasks in parallel, eliminating the need to manage code or infrastructure because of changes in demand. For my specific use case, I achieved parallel execution for 1000 UOW with a completion time of fewer than 23 seconds in total. This use-case demonstrates the power of lambdas within the serverless world, which is to scale to hundreds of instances without any manual intervention required to scale up or down the infrastructure.
The serverless world provides us with a higher level of abstraction than ever, allowing us to design and build innovative yet simple, effective, and scalable solutions in response to complex problems.