A decoupled Service-Oriented Architecture is the blueprint for a microservices architecture. So microservices applications are designed and developed as a collection of loosely coupled services with private databases each per service. But then you may use different SQL or NoSQL databases for each service as per the kind of data you store and process within each service. That means you cannot simply apply local ACID transactions to retrieve data in different databases in this decoupled, distributed architecture.
This leads to challenges in distributed data management such as, how to maintain data consistency across multiple services and how to implement queries to retrieve data distributed across multiple heterogeneous databases. The solution is event-driven architecture.
Unlock the future of intelligent applications with our cutting-edge Generative AI integration services!
What is Event-Driven Architecture?
In an event-driven architecture, events trigger the services or components within the infrastructure. An event can be a change or an update in the state such as the change of state of an EC2 instance from ‘running’ to ‘stopped’ or a new order being placed in an e-commerce website, etc.
An event-driven architecture implemented in AWS typically has three components as a producer, an event router, and consumers. A producer is a service that publishes or pushes an event to the event router when something notable happens, such as a change in the business entity. The event router then filters the published events and routes the outcome to the consumers. The event router by sitting in between the event producer and the consumers preserves the decoupled architecture, improving application resiliency and developer agility in the microservices architecture.
You can use Amazon SNS (Simple Notification Service), Amazon EventBridge, Amazon Lambda, and Amazon Kinesis event-driven services in AWS as event routers to develop event-driven microservices applications.
Event-Driven with Amazon EventBridge
Amazon EventBridge is a serverless event bus service that receives events from event producers. Amazon Eventbridge can receive events from the AWS services in the AWS environment, SaaS partner services or applications, and your own custom (micro)services or applications. Then, EventBridge event buses using the routing rules attached to each event bus, route the events to targets such as Kinesis streams, Amazon SNS topics, Lambda functions, Amazon SQS queues, etc. to process the events.
For example, you can create an “Orders” event bus applying an “OrdersDevRule” that matches all the events sent from a custom “Order” (micro)service you created in your e-commerce application to send to a CloudWatch Logs log group. You can find a step-by-step guide for this use case here.
Event-Driven with Amazon SNS
REST-based HTTP/S protocol is the most popular method to implement synchronous communication in microservices. But, to implement asynchronous messaging and event passing, the ideal solution from AWS is the Amazon SNS with Amazon SQS (Simple Queue Service).
Amazon SNS like Amazon EventBridge can act as a message broker that allows applications to push event notifications to subscribers. When you use Amazon SNS with the Amazon SQS (to validate message delivery), you can send one message to multiple consumers.
For example, consider an inventory tracking system that should keep track of stock in a warehouse. When the inventory tracking system changes its state, such as when the stock depletes over the minimum limit and stocks sell out completely, these notifications publish to an SNS topic. The SNS topic has subscribed with SQS queues connected to consumer EC2 instances (services). Thus, whenever a change is published to the SNS topic, Amazon SNS sends a message to the subscribed SQS queues to learn about the changes and update their databases accordingly to process the incoming business transactions. You can learn how to create an Amazon event-driven architecture with SNS FIFO from here.
Event-Driven with Amazon Lambda
You can develop microservices as Lambda-based applications. Lambda-based applications run custom code in response to the events following many of the best practices in event-driven architectures. Lambda functions process events.
As databases in a microservices architecture cannot leverage ACID transactions, some business transactions may end up as partial executions. So in event-driven architectures, you should use a distributed Saga pattern to undo the changes preceding a failed transaction. You can implement a Saga execution coordinator with AWS Step Functions. Along with that, you can use AWS Lambda with scheduled Amazon CloudWatch events to build a cleanup and deduplication mechanism.
SERVICE DISABLED VETERAN OWNED SMALL BUSINESS (SDVOSB)
AWS Lambda also has AWS Lambda Destinations that you can use to route asynchronous function outcomes to destination resources such as another Lambda function, EventBridge, SNS, or SQS without needing to write extra code.
In addition to the Lambda destinations, Lambda offers Dead Letter Queues for function execution failure handling. Here you can find an example with Lambda Destinations and DLQ.
Event-Driven with Amazon Kinesis Stream
If low latency is a key requirement for your application, Amazon Kinesis is a better solution to ingest and process incoming events. You can scale up or down your Streams by adding or removing shards with Amazon Kinesis.
Linking Amazon Kinesis Stream to send events and lambda functions to execute the events and update the database in each (micro)service is a way you can achieve the event-driven architecture with event producers hosted in AWS. You can reduce the latency that can often be experienced with Amazon Lambda when executing events followed by 5-minute gaps when you use a Kinesis Stream to pass the events. Then the latency to process an event is reducible to 3 to 4 seconds (when using NodeJS) for an event following after a 5-minute gap the previous. However, you may still incur higher latencies if using heavy languages such as C# or Java.
Small Disadvantaged Business
Small Disadvantaged Business (SDB) provides access to specialized skills and capabilities contributing to improved competitiveness and efficiency.
Final Thoughts
An event driven architecture has many benefits, including the flexibility to span multiple services and achieve eventual consistency while being able to maintain materialized views across the application. However, the well-known drawback in event-driven architectures is the complex programming model. The above-mentioned AWS event-driven services mightily simplify the development complexity of microservices-based on the event-driven architecture in serverless and cloud-native environments.
Further blogs within this Event-Driven Architecture category.