Event Driven Microservices

🧠 Free Recall ❓ Quiz 🔍 Cues 🔮 Simulation 🥷 ShadowBox 📐 Encode Cards

Eric Evans’s book Domain-Driven Design helped us understand the importance of representing the real world in our code, and showed us better ways to model our systems.The overarching philosophy of DDD is to use the notion of bounded contexts which form protective layers around models that define the business domain. Bounded contexts are analogous to departments in a company — the legal department has certain specific responsibilities (contexts) that are different than the IT department and those responsibilities are enforced by rules (boundaries) for interaction and obtaining services from the departments. Microservices is an implementation approach that encourages you to focus your service boundaries on the business domain boundaries. DDD and microservices can be used together as you move your organization to a service-oriented design and reap the benefits of continuous integration and delivery thus facilitating delivery of true business value through easily compossible applications and APIs by automating repeatable business processes.The microservice architecture structures an application as a set of loosely coupled services thus accelerating software development.

Characteristics of Micro services
Service decomposition

A monolithic server is a natural way to approach building such a system. All your logic for handling a request runs in a single process, allowing you to use the basic features of your language to divide up the application into classes, functions, and namespaces. With some care, you can run and test the application on a developer’s laptop, and use a deployment pipeline to ensure that changes are properly tested and deployed into production. You can horizontally scale the monolith by running many instances behind a load-balancer. problem is over time it’s often hard to keep a good modular structure, making it harder to keep changes that ought to only affect one module within that module.
The main idea of a micro service architecture is to take a monolithic architecture and decomposing it .In short, the micro service architectural style is an approach to developing a single application as a suite of small services thus promoting modularity there are multiple options for deciding how to decompose your monolith each having its pro and cons
Decompose by Business capability
Understanding the core components of the business (e.g., messaging, logging, user-interface, etc.)
Build a service for each capability
N.B: You need to understand the core components very well.
Decompose by Sub Domain
further decomposition of the business services can be done based on subdomain
Decompose by locality
Sometimes you will find that there are some services that communicate more with one another compared to other services.
Find the services that communicate intensely with one another and reason if the can be merged as one service.
Keep services that communicate less in separate components
N.B: you need to understand the weight of the links between each of your sub-components. If the weight is too high, then consider having them as one service component.

Horizontal Scaling
Horizontal scaling means that you scale by adding more machines into your pool of resources whereas Vertical scaling means that you scale by adding more power (CPU, RAM) to an existing machine.the benefits of scaling out in a cloud environment include:

Instant and continuous availability
No limit to hardware capacity
Cost can be tied to use
You’re not stuck always paying for peak demand
Built-in redundancy
Easy to size and resize properly to your needs
Allows for autoscaling by automagically adjusting capacity to maintain steady, predictable performance at the lowest possible cost

Service Isolation
Isolation is the most important trait of Microservices . It is the foundation for many of the high-level benefits in Microservices. But it is also the trait that has the biggest impact on your design and architecture. It will, and should, slice up the whole architecture, and therefore it needs to be considered from day one. generally the advantage of service isolation hits home when doing service composition to model preexisting business processes and will impact how efficiently your business runs .
As hypothesized by Conway’s law Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure. so how you define your service architecture will often reflect how you communicate within a company
Isolation also makes it easier to scale each service, as well as allowing them to be monitored, debugged and tested and deployed independently — some‐ thing that is very hard if the services are all tangled up in the big bulky mess of a monolith.
Isolation also makes it easier to isolate faulty systems and fix problems with minimum to little downtime
Fault Tolerence
Microservices depend on other services in order to fulfill their requirements. Since microservices are deployed independently , their instances may go up and down frequently. Also since they are deployed on commodity servers you will always have servers failing .
One of the best advantages of a microservices architecture is that you can isolate failures and achieve graceful service degradation as components fail separately. For example, during an outage customers in a photo sharing application maybe cannot upload a new picture, but they can still browse, edit and share their existing photos.
We also want our components to fail fast as we don’t want to wait for broken instances until they timeout.Another need is for the system to be self-healing to recover an application and its state on failure. The good thing about microservices though is that it uses containerization for deployment .The use of containerization provides the advantage of quickly being able to spin up new containers on another server or VM if one does go down .
With microservices You have more services to monitor for availability and performance. It’s also a challenge to balance load and route around failures to maintain high availability.
The problems of resiliency are compounded during scenarios, such as when failures occur during an application upgrade. The microservice, working with the deployment system, needs to determine whether it can continue to move forward to the newer version or instead roll back to a previous version to maintain a consistent state.
Smart Endpoints and Dumb pipes
The idea of “smart endpoints, dumb pipes” is that you should minimize the intelligence in your infrastructure, putting more of an onus on the services to implement their own functional, and non-functional capabilities; and on the service consumers to understand how to consume each of the microservices. This is essentially the ESB anti-pattern, and is a result of too much dependence on ESBs for non-functional requirements, and in some cases even for business logic.
Stateless Services
The interaction of any two software programs involves keeping track of the interaction-specific data as each subsequent interaction may depend upon the outcome of the previous interaction.In a service composition, a service may need to store activity-specific data in memory while it is waiting for another service to complete its processing. Consequently, in case of service-orientation, an efficient management of service activity related data becomes more important as service-orientation puts a lot of emphasis on service reuse.
the problem arises though when the transactional state is stored on the same server executing the code , due to this storage of state in a specific location service requests are often required to be Sticky ( that is one a request has been made from one server and a server session is created all subsequent calls need to go to the same server ) this limits the system scalability and overall uptime because if the server or load balancer is down this may have adverse effect on performance and availability.
so one option to solve this problem is to make the service stateless .this reduces the need to go to a particular server for a particular request but the state still needs to be stored somewhere and is generally abstracted away into shared database instance from where it can be restored .
Decentralized data management
Microservices prefer letting each service manage its own database, either different instances of the same database technology, or entirely different database systems — an approach called Polyglot Persistence. You can use polyglot persistence in a monolith, but it appears more frequently with microservices.
Infrastructure automation
Part of creating microservices architecture involves developing infrastructure automation, container management solutions, and a DevOps pipeline for microservices. Docker and container management solutions increases infrastructure resource utilization, speeds up development, and decouples app layer from infrastructure. Infrastructure automation by setting up a CI/CD pipeline helps free development resources to work on value creation
In summary the micro service architecture provides the following advantages and disadvantages .
Advantages Of Micro Services
Easier Deployments

Technology Diversity
More Frequent and Independent deployments
Isolation
Fault tolerance

Pattern 1: Loosely Coupled , Reactive and Event driven services
A Service is a closely related set of operations that are encapsulated together. Generally the service is independent of other services i.e. the service does something useful and discrete in terms of the business process. To model and perform the overall business process, a set of services must be assembled into an overall service environment. Looking from the perspective of the type of service under consideration, you could have

1 ) Utility services (Infrastructure services) — usually defined by grouping together infrastructural capabilities with a common purpose, for example a Logging Service, and thus granularity is directly defined by the utility functions supported;
2 ) Entity services — these have their functional context scoped by the entities that they manage.
3 ) Task services — these typically contain a group of capabilities related to the same business task.
4 ) Process services — these usually deal with a larger functional context defined by the encapsulated business process.
Things connect to other things and applications for a variety of reasons. Licensing, activation, sending and receiving data, updating, patching, etc… They connect to perform a specific and sometimes very granular set of functions.IoT platforms therefore require services that are a specialized type of proxy service and stand in front of a hardware device. They need to provide provision to control and communicate with devices that do not natively provide an API that is compatible with the protocols used throughout the system.this service thus translates between the machine’s communication channel(s) and those protocols required to converse with other upstream and downstream systems this functionality is generally provided with the help of device SDKs and Device gateways.
besides this in an environment where multiple devices having the sensors of the same time can be present value could also be found by working with sensors at a more granular model very important step in starting your IoT transformation journey is to figure out where in the SOA maturity model do you currently stand as a visual tool to help your evaluation see table below . My advice don’t start your IoT transformation before you reach at least a level 3 preferably a level 4.

As seen in the above service maturity model level 3 maturity calls for componentization of the monolithic 3 tier architecture .componentization gives the ability to scale through cloning and replication, separating functionality or services, and splitting similar data sets across storage and application systems.
Microservice based
Microservices are therefore the best architectural option for the Internet of Things because the decomposition of components on the basis of the function the “Thing” wants to achieve provides a clear separation of concerns and isolation . Also both microservices IoT work on the concept of smart endpoints and dumb pipes , and the emphasis of the microservices architectural style on business processes instead of architectural components and layer further aids in mapping business process to devices and respond quickly to events .Ultimately, a microservice works best when it has one, and only one, responsibility — you can even wrap most of your third party dependencies to make sure you don’t have to think about them in other parts of the app.while I can go on about the advantages and disadvantages of using this architecture pattern it is beyond the scope of this article and will be explored later in a separate article .
Reactive
Building microservices is relatively easy. The more difficult part is everything that happens between the services in distributed systems.the buggiest problem of having n-tier architectures is that with large number of systems communicating with each other a deep understanding of distributed systems is required . The microservice architecture has several inherent flaws like
Tight Coupling of Systems

2. Dependency on the network performance for system performance
3. Availability of system dependent on availability of each individual component
4. A generally large volume of service calls and data transport requiring the need to be able to handle back pressure and durability
Also IoT devices often encounter intermittent connectivity in their wireless networks. Also, devices sometimes suspend their radio interfaces or hibernate entirely in order to conserve power. Moreover, the set of devices deployed in the field changes with time: A temperature sensor may not know who the exact recipient of its telemetry information is. It may be one or more existing thermostats, or new thermostats and services in the future.
This further adds to the need for the durability of the system Considering these architectural drawbacks it is essential to decouple the micro services through distributed public subscribe queueing systems like Kafka .
Eventdriven
another salient feature of IoT platform is that it should be able to react to triggering events for the devices in real time and at scale this tends to fit well with The event-driven architectures.
The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events in IoT systems . A key element of event notification is that the source system doesn’t really care much about the response .The system must be able to ingest the data at the volume and throughput that is required by the data source, multiple instances might also be necessary to handle the volume and frequency of events.
The event-driven architecture pattern consists of two main topologies, the mediator and the broker. The mediator topology is commonly used when you need to orchestrate multiple steps within an event through a central mediator, whereas the broker topology is used when you want to chain events together without the use of a central mediator.
While I have spent some time thinking about what event architecture pattern to use in my architecture turns out the best answer that I can come up is it depends , one important criteria of being whether you care about transactional atomicity or not .
IoT services are therefore required to be Loosely Coupled , Reactive and Event driven services

Benefits of an event-driven architecture approach
By Scott Heather, Bits In Glass
While event-driven architecture is not a new digital transformation approach, it’s become more popular over the last few years thanks to the growth of Internet of Things (IoT) devices — like cameras, sensors, and wearables — and artificial intelligence (AI).
Check out my last blog post to refresh your memory on what an event-driven approach is and where it can be used. In one sentence, event-driven architecture analyzes how events are occurring and changing throughout your business operations in real-time, so your people or machines can collaborate on the best actions.
Event-driven architecture is becoming a popular choice for modernizing, automating, and bringing intelligence into business operations and processes. So why should you consider an event-driven approach as part of your digital transformation journey?
Better human agility
As technology continues to grow and evolve, and software systems get more intelligent and complex, our human intuition, experience, and problem-solving is still an important aspect in addressing situations and resolving conflicts.
David Cearley, a Distinguished VP Analyst with Gartner says, “Rather than building a technology stack and then exploring the potential applications, organizations must consider the business and human context first.”
What’s great about event-driven architecture is that it makes it easy to build applications that enable software systems, mechanical systems (like robots), and people to work together in complex scenarios.
Nowadays, many would say thanks to Amazon, we want to do whatever we want immediately, get instant feedback on the status of what we did, and be able to have real-time interaction with anybody involved in what we’ve done.
An event-driven approach serves this mentality by building applications based on events occurring within an organization’s operations. It also improves human agility by equipping employees with real-time data and information so they can quickly step in, easily understand what’s happening, and respond appropriately.
Improved IoT alignment
As I mentioned above, the rise of event-driven architecture can be tied to the growth and rising popularity of IoT (Internet of Things) devices. Statista, an industry research firm, expects the global IoT market to grow from $164 billion in 2018 to $1.6 trillion by 2025.
Organizations are adding IoT devices like sensors, cameras, and smart machines into their operations to better track, monitor, and deliver business. In healthcare for example, hospitals can use IoT sensors to monitor nurse entry into patient rooms, enable instant staff notifications for codes, generate “out of bed” patient alerts, alert staff of inappropriate patient movements, and track the location and use of hospital assets.
For IoT interaction like this to work effectively and efficiently, applications need to be built to transfer activity data from the IoT device to its processing center and out to consuming applications and devices – enabling real-time information and action. This is where event-driven architecture comes in!
An event-driven approach takes your operations beyond systems and dashboards and adds in real-time data streams for improved actions and responses. Using one of the examples above, an event-driven application would take sensor data from a hospital bed of a patient who needs to remain in bed and immediately send notifications to the on-duty nursing staff alerting them that the patient has gotten out of bed. 
Taking AI to the next level
McKinsey & Company estimate that artificial intelligence (AI) has the potential to create up to $5.8 trillion in annual value across nine business functions in 19 industries. We know that many industries are already adopting AI to help process information and provide decision-making data. Within healthcare, providers can use AI to help improve the speed and accuracy of diagnoses, and within financial services to predict customer behaviors and needs to better serve them. 
The value of AI data and information is directly impacted by how quickly it’s acted on. An event-driven approach enables powerful applications to be created using AI and ensures action is taken in real-time. For example, Augusta Health has saved 282 lives with an AI-infused sepsis early warning system. They used AI to monitor patients at risk for potential infections or sepsis, and an event-driven application to immediately respond and alert the appropriate staff when patient vitals reach critical levels. 
This approach allows for real-time responses, versus the traditional approach of nursing staff checking and responding to patient vitals on an hourly basis. And as a result of their event-driven and AI-infused system, the sepsis mortality rate at August Health is now 4.8%, compared to 13.2% at the state level.
Enables digital twins
A digital twin is a digital representation of an asset. The asset can be physical, like a building, machine, or camera, or it can be logical, like a business process or service. By representing these environments digitally we can monitor and control things in new and innovative ways. Digital twins improve operational efficiency, safety, and security, and reduce cost, waste, and redundancy.
Back in late 2019, Benoit Lheureux, research vice president at Gartner said “We predicted that by 2022, over two-thirds of companies that have implemented IoT will have deployed at least one digital twin in production. We might actually reach that number within a year.” 
If we look at exploration and production within the energy industry for example, by creating a digital twin of a company’s oil rig, the amount of data sent to the cloud will be reduced, cutting out unnecessary data transfers, improving cybersecurity, and decreasing network and system response times. As a result, analysis and response to critical processes and production are now up to date and completed in real-time.
An event-driven approach connects your digital twins and assets, seamlessly transfers information between them all in real-time, and includes machine-man collaboration so humans can step in and respond when needed.
So what have we learned? An event-driven approach has a number of benefits, from better human agility through improved machine-man collaboration, improved IoT interaction and alignment, taking AI to the next level, and enabling digital twins infrastructure. By focussing on how your operations occur in real-time, event-driven architecture enables you to be better equipped to meet needs, respond effectively, and grow your business. 
Learn more about event-driven architecture in VANTIQ’s whitepaper on The Modern Event-Driven Architecture Landscape here.

How to write event-driven IoT microservices that don’t break
Today’s IoT environments are a potential rat’s nest of asynchronous craziness just waiting to happen

Thinkstock
“Asynchrony” is a scary word. It means taking events as they come, managing somehow to avoid being overtaken by them.
Event-driven asynchrony is the foundation of serverless computing, which, as a programming framework, is tailor-made for the internet of things. When you consider event scenarios in an IoT context, the chief drivers are the never-ending stream of sensor inputs that—depending on their timing, sequencing, frequency, and values—can swing the runtime behavior of the system arbitrarily in every possible direction.
When you layer event-driven microservices interactions over these sensor-driven complexities, it’s clear that today’s IoT environments are a potential rat’s nest of asynchronous craziness just waiting to happen.
[ Also on InfoWorld: Tim O’Reilly: The golden age of the programmer is over ]
Bringing resiliency to distributed IoT microservices requires a high-level programmatic abstraction for keeping its fundamentally asynchronous substrate under control. That’s how I’m interpreting the recent InfoWorld article about an open-source programming language called P that Microsoft has introduced for programming asynchronous applications in embedded systems, AI applications, and cloud services.
One concept that jumped out at me was the notion of a “heisenbug,” which the article defines as “timing-related bugs that often disappear during an investigation of it.” The term “heisenbug” stems from the analogy of physics’ Heisenberg Uncertainty Principle, under which the attempt to observe a system inevitably alters its state.
Where computing environments are concerned, heisenbugs are equivalent to probe effects, in which attaching a test probe—or simply sending an asynchronous test ping—to a system changes its behavior. What that implies is that the very act of trying to isolate, analyze, and debug some systemic glitches will alter the underlying systemic behavior of interest—perhaps causing the bugs in question not to recur. One of the chief causes of heisenbugs are race conditions, under which a system behaves erratically when asynchronous input events don’t take place in the specific order expected by that system’s controlling program.
What programmer can possibly predict—much less write code sequences to deal with—every possible scenario of event-driven asynchronous interactions in the IoT and other fundamentally asynchronous computing environments? That’s why P, or an equivalent safe asynchronous event-driven programming language, is essential and should be a required tool in every IoT or microservices developer’s toolkit.
RECOMMENDED WHITEPAPERS
Fixed Network Foundation Layer for 21st Century Smart Buildings
Fixed Network Foundation Layer for 21st Century Smart Buildings
Elmhurst Hospital Case Study
Such languages let developers specify the precise sequence of execution steps to be taken by compiled code, regardless of the order in which the underlying runtime environments process asynchronous event messages. Programmers build application logic using abstraction layers that generate asynchronous event-handling code as collections of interacting state machines. What these state machines do is defer the handling of specific events until their proper place in sequence.
It’s hard to see how distributed IoT-based microservices, such as the Iota distributed hyperledger blockchain, can be hardened without this sort of safe event-driven programming language. And it’s unclear how IoT’s RESTful APIs can offer predictable real-time response and programming efficiency without the ability to enforce distributed state transitions among distributed services interoperating over a loosely coupled event-driven fabric.
Of course, there will always be bugs galore in a complex distributed system such as the IoT. However, it will be easier to isolate bugs in the IoT’s underlying runtime platforms—such as the smart sensors that power the industrial Internet—if developers can leverage a programming environment that eliminates any glitches may otherwise spring from IoT’s asynchronous integration fabric.

Unstarred

Dunk (exclude from study)

Event Driven Microservices

Review Questions

Todos

Sensemaker