Load Balancer

What is a load-balaner?

A load balancer is a software or hardware device that keeps any one server from becoming overloaded. A load-balancing algorithm is a logic that a load balancer uses to distribute network traffic between servers (an algorithm is a set of predefined rules).

Dynamic load balancing algorithms

Last connection: Checks which servers have the fewest connections open at the time and send traffic to those servers. This assumes all connections require roughly equal processing power.

Weighted least connection: Gives administrators the ability to assign different weights to each server, assuming that some servers can handle more connections than others.

Weighted response time: Averages the response time of each server, and combines that with the number of connections each server has open to determine where to send traffic. By sending traffic to the servers with the quickest response time, the algorithm ensures faster service for users.

Resource-based: Distributes load based on what resources each server has available at the time. Specialized software (called an "agent") running on each server measures that server's available CPU and memory, and the load balancer queries the agent before distributing traffic to that server.

Static load balancing algorithms

Round robin: Round robin load balancing distributes traffic to a list of servers in rotation using the Domain Name System (DNS). An authoritative nameserver will have a list of different A records for a domain and provides a different one in response to each DNS query.

Weighted round robin: Allows an administrator to assign different weights to each server. Servers deemed able to handle more traffic will receive slightly more. Weighting can be configured within DNS records.

IP hash: Combines incoming traffic's source and destination IP addresses and uses a mathematical function to convert it into a hash. Based on the hash, the connection is assigned to a specific server.

CAP Theorem in Distributed System

You cannot combine these three features all together “Cheap, Fast, and Good.

The CAP theorem applies a similar type of logic to distributed systems—namely, that a distributed system can deliver only two of three desired characteristics: consistency, availability, and partition tolerance (the ‘C,’ ‘A’ and ‘P’ in CAP).

A distributed system is a network that stores data on more than one node (physical or virtual machines) at the same time. Because all cloud applications are distributed systems, it’s essential to understand the CAP theorem when designing a cloud app so that you can choose a data management system that delivers the characteristics your application needs most.

Let’s take a detailed look at the three distributed system characteristics to which the CAP theorem refers.

Consistency

Consistency means that all clients see the same data at the same time, no matter which node they connect to. For this to happen, whenever data is written to one node, it must be instantly forwarded or replicated to all the other nodes in the system before the write is deemed ‘successful.’

Availability

Availability means that any client making a request for data gets a response, even if one or more nodes are down. Another way to state this—all working nodes in the distributed system return a valid response for any request, without exception.

Partition tolerance

A partition is a communications break within a distributed system—a lost or temporarily delayed connection between two nodes. Partition tolerance means that the cluster must continue to work despite any number of communication breakdowns between nodes in the system. 

Distributed Logging and Tracing in Microservices

The microservice architecture pattern. Requests often span multiple services. Each service handles a request by performing one or more operations, e.g. database queries, publishes messages, etc.

How to understand the behavior of an application and troubleshoot problems?

The solution is Distributed Tracing and Log Aggregation

Distributed Tracing

Trace ID: A unique ID to trace the path of a request, If the request spans multiple services (Request ID).

Span ID: A unique ID to track requests related to a single service only (Service ID).

Log Aggregation

Storing of logs for visualization and analysis in an external/centralized place. Additional infra and setup costs. How to implement Log Aggregation.

1) Spring Cloud Sleuth + Zepkin

2) OpenTracing + Jaeger

3) ELK (ElasticSearch, Logstash, Kibana)

Note:

It is not mandatory to introduce Log Tracing in every Microservice architecture.

It is used for monitoring the behavior of applications and triggering specific alerts based on errors or exceptions. It totally depends on:

1) How you have designed your Architecture of application?

2) The microservices you created are independent in terms of resource processing or not?

Saga Pattern for Distributed Transaction Management in Microservices

One of the benefits of microservice architecture is that we can choose the technology stack per service. For instance, we can decide to use a relational database for service A and a NoSQL database for service B.
This model lets the service manage domain data independently on a data store that best suits its data types and schema. Further, it also lets the service scale its data stores on demand and insulates it from the failures of other services.
However, at times a transaction can span across multiple services, and ensuring data consistency across the service database is a challenge.

Implement each business transaction that spans multiple services as a saga. A saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.

There are two ways of coordinating sagas:

• Choreography - Event-Based (Message Broker)
• Orchestration - Command-Based (Service Provider)

Choreography-based saga

1. The Order Service receives the POST /orders request and creates an Order in a PENDING state
2. Then Order Service publishes an event to the message broker that an ORDER_CREATED to Payment Service.
3. The Payment Service got the ORDER_CREATED event and does the necessary updates and publishes the event to Order and Restaurant Service that an ORDER_PAID.
4. The Order and Restaurant Service got the ORDER_PAID event and do the necessary updates.
5. The Restaurant Service publishes the event to Order and Delivery Service that an ORDER_PREPARED.
6. The Order and Delivery Service got the ORDER_PREPARED event and do the necessary updates.
7. Finally, Delivery Service publishes the event to Order Service that ORDER_DELIVERED to the Order Service, and the Order state changed from PENDING to COMPLETE.

Orchestration-based saga

1. The Order Service receives the POST /orders request and creates an Order in a PENDING state
2. Then Order Service sends the command to Orchestrator Service that an ORDER_CREATED to Payment Service.
3. The Payment Service got the ORDER_CREATED command and does the necessary updates and sends the command to Order and Restaurant Service that an ORDER_PAID.
4. The Order and Restaurant Service got the ORDER_PAID command and do the necessary updates.
5. The Restaurant Service sends the command to Order and Delivery Service that an ORDER_PREPARED.
6. The Order and Delivery Service got the ORDER_PREPARED command and do the necessary updates.
7. Finally, Delivery Service sends the command to Order Service that ORDER_DELIVERED to the Order Service, and the Order state changed from PENDING to COMPLETE.

Conclusion

CHOREOGRAPHY-Event-Based (Message Broker) could be a case where you can end in a deadlock because one service is dependent on the other and what if the other is dependent on the other, so if you have a very less number of events it will be very good but if let's say you have more number of events there could be a point where you might miss the sequence of the workflow and it might be tedious to test individual microservices and integration tests when you have too many events, in terms of ORCHESTRATION Command-Based (Service Provider) there is a single service which is going to interact with different microservices and now orchestration needs to have mapping on what each service does which again defeats the purpose of individual microservices concept where your orchestration service knows what is service does and if let's say there is a change is a payment service it might end up the changing the orchestration service as well. So this is one more pattern of implementing SAGAS however it's up to you to decide which one to go for based on your architectural use case.