Load Balancer

What is a load-balaner?

A load balancer is a software or hardware device that keeps any one server from becoming overloaded. A load-balancing algorithm is a logic that a load balancer uses to distribute network traffic between servers (an algorithm is a set of predefined rules).

Dynamic load balancing algorithms

Last connection: Checks which servers have the fewest connections open at the time and send traffic to those servers. This assumes all connections require roughly equal processing power.

Weighted least connection: Gives administrators the ability to assign different weights to each server, assuming that some servers can handle more connections than others.

Weighted response time: Averages the response time of each server, and combines that with the number of connections each server has open to determine where to send traffic. By sending traffic to the servers with the quickest response time, the algorithm ensures faster service for users.

Resource-based: Distributes load based on what resources each server has available at the time. Specialized software (called an "agent") running on each server measures that server's available CPU and memory, and the load balancer queries the agent before distributing traffic to that server.

Static load balancing algorithms

Round robin: Round robin load balancing distributes traffic to a list of servers in rotation using the Domain Name System (DNS). An authoritative nameserver will have a list of different A records for a domain and provides a different one in response to each DNS query.

Weighted round robin: Allows an administrator to assign different weights to each server. Servers deemed able to handle more traffic will receive slightly more. Weighting can be configured within DNS records.

IP hash: Combines incoming traffic's source and destination IP addresses and uses a mathematical function to convert it into a hash. Based on the hash, the connection is assigned to a specific server.

CAP Theorem in Distributed System

You cannot combine these three features all together “Cheap, Fast, and Good.

The CAP theorem applies a similar type of logic to distributed systems—namely, that a distributed system can deliver only two of three desired characteristics: consistency, availability, and partition tolerance (the ‘C,’ ‘A’ and ‘P’ in CAP).

A distributed system is a network that stores data on more than one node (physical or virtual machines) at the same time. Because all cloud applications are distributed systems, it’s essential to understand the CAP theorem when designing a cloud app so that you can choose a data management system that delivers the characteristics your application needs most.

Let’s take a detailed look at the three distributed system characteristics to which the CAP theorem refers.

Consistency

Consistency means that all clients see the same data at the same time, no matter which node they connect to. For this to happen, whenever data is written to one node, it must be instantly forwarded or replicated to all the other nodes in the system before the write is deemed ‘successful.’

Availability

Availability means that any client making a request for data gets a response, even if one or more nodes are down. Another way to state this—all working nodes in the distributed system return a valid response for any request, without exception.

Partition tolerance

A partition is a communications break within a distributed system—a lost or temporarily delayed connection between two nodes. Partition tolerance means that the cluster must continue to work despite any number of communication breakdowns between nodes in the system. 

Distributed Logging and Tracing in Microservices

The microservice architecture pattern. Requests often span multiple services. Each service handles a request by performing one or more operations, e.g. database queries, publishes messages, etc.

How to understand the behavior of an application and troubleshoot problems?

The solution is Distributed Tracing and Log Aggregation

Distributed Tracing

Trace ID: A unique ID to trace the path of a request, If the request spans multiple services (Request ID).

Span ID: A unique ID to track requests related to a single service only (Service ID).

Log Aggregation

Storing of logs for visualization and analysis in an external/centralized place. Additional infra and setup costs. How to implement Log Aggregation.

1) Spring Cloud Sleuth + Zepkin

2) OpenTracing + Jaeger

3) ELK (ElasticSearch, Logstash, Kibana)

Note:

It is not mandatory to introduce Log Tracing in every Microservice architecture.

It is used for monitoring the behavior of applications and triggering specific alerts based on errors or exceptions. It totally depends on:

1) How you have designed your Architecture of application?

2) The microservices you created are independent in terms of resource processing or not?

Saga Pattern for Distributed Transaction Management in Microservices

One of the benefits of microservice architecture is that we can choose the technology stack per service. For instance, we can decide to use a relational database for service A and a NoSQL database for service B.
This model lets the service manage domain data independently on a data store that best suits its data types and schema. Further, it also lets the service scale its data stores on demand and insulates it from the failures of other services.
However, at times a transaction can span across multiple services, and ensuring data consistency across the service database is a challenge.

Implement each business transaction that spans multiple services as a saga. A saga is a sequence of local transactions. Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga. If a local transaction fails because it violates a business rule then the saga executes a series of compensating transactions that undo the changes that were made by the preceding local transactions.

There are two ways of coordinating sagas:

• Choreography - Event-Based (Message Broker)
• Orchestration - Command-Based (Service Provider)

Choreography-based saga

1. The Order Service receives the POST /orders request and creates an Order in a PENDING state
2. Then Order Service publishes an event to the message broker that an ORDER_CREATED to Payment Service.
3. The Payment Service got the ORDER_CREATED event and does the necessary updates and publishes the event to Order and Restaurant Service that an ORDER_PAID.
4. The Order and Restaurant Service got the ORDER_PAID event and do the necessary updates.
5. The Restaurant Service publishes the event to Order and Delivery Service that an ORDER_PREPARED.
6. The Order and Delivery Service got the ORDER_PREPARED event and do the necessary updates.
7. Finally, Delivery Service publishes the event to Order Service that ORDER_DELIVERED to the Order Service, and the Order state changed from PENDING to COMPLETE.

Orchestration-based saga

1. The Order Service receives the POST /orders request and creates an Order in a PENDING state
2. Then Order Service sends the command to Orchestrator Service that an ORDER_CREATED to Payment Service.
3. The Payment Service got the ORDER_CREATED command and does the necessary updates and sends the command to Order and Restaurant Service that an ORDER_PAID.
4. The Order and Restaurant Service got the ORDER_PAID command and do the necessary updates.
5. The Restaurant Service sends the command to Order and Delivery Service that an ORDER_PREPARED.
6. The Order and Delivery Service got the ORDER_PREPARED command and do the necessary updates.
7. Finally, Delivery Service sends the command to Order Service that ORDER_DELIVERED to the Order Service, and the Order state changed from PENDING to COMPLETE.

Conclusion

CHOREOGRAPHY-Event-Based (Message Broker) could be a case where you can end in a deadlock because one service is dependent on the other and what if the other is dependent on the other, so if you have a very less number of events it will be very good but if let's say you have more number of events there could be a point where you might miss the sequence of the workflow and it might be tedious to test individual microservices and integration tests when you have too many events, in terms of ORCHESTRATION Command-Based (Service Provider) there is a single service which is going to interact with different microservices and now orchestration needs to have mapping on what each service does which again defeats the purpose of individual microservices concept where your orchestration service knows what is service does and if let's say there is a change is a payment service it might end up the changing the orchestration service as well. So this is one more pattern of implementing SAGAS however it's up to you to decide which one to go for based on your architectural use case.

Java 9 Module System

Java 9 introduces a new level of abstraction above packages, formally known as the Java Platform Module System (JPMS), or “Modules” for short.

Java 9 modules are one of the biggest changes in the structure of Java.

What is Module?

A Module is a group of closely related packages and resources along with a new module descriptor file (module-info.java).

In other words, it's a “package of Java Packages” abstraction that allows us to make our code even more reusable.

Problems of Current Java System?

"Why we need Java SE 9 Module System” means the problems of the Current Java SE8 or earlier System.

• As JDK is too big, it is a bit tough to scale down to small devices. Java SE 8 has introduced 3 types of compact profiles to solve this problem: compact1, compact2, and compact3. But it does not solve this problem.
• JAR files like rt.jar etc are too big to use in small devices and applications.
• There is no Strong Encapsulation in the current Java System because the “public” access modifier is too open. Everyone can access it.
• As JDK, JRE is too big, it is hard to Test and Maintain applications.
• As the public is too open, They are not to avoid accessing of some Internal Non-Critical APIs like the sun.*, *.internal.*, etc. Security is also a big issue.

Advantages of Java SE 9 Module System

Java SE 9 Module System is going to provide the following benefits

• As Java SE 9 is going to divide JDK, JRE, JARs, etc, into smaller modules, we can use whatever modules we want. So it is very easy to scale down the Java Application to Small devices.
• Ease of Testing and Maintainability.
• As the public is not just public, it supports very Strong Encapsulation.
• We cannot access Internal Non-Critical APIs anymore.
• Modules can hide unwanted and internal details very safely, and we can get better Security.
• The application is too small because we can use only whatever modules we want.
• It's easy to support Single Responsibility Principle (SRP).

Compare JDK 8 and JDK 9

We know what JDK software contains. After installing JDK 8 software, we can see a couple of directories like bin, jre, lib etc in Java Home folder. However, Oracle Corp has changed this folder structure a bit differently as shown below.

Mother of Java 9 Module System

As of now, Java 9 Module System has 95 modules in Early Access JDK. Oracle Corp has separated JDK jars and Java SE Specifications into two sets of Modules.

• All JDK Modules start with “jdk.*”
• All Java SE Specifications Modules start with “java.*”

Java 9 Module System has a “java.base” Module. It’s known as Base Module. It’s an Independent module and does NOT dependent on any other modules. By default, all other Modules are dependent on this module. That’s why “java.base” Module is also known as The Mother of Java 9 Modules. It’s the default module for all JDK Modules and User-Defined Modules just like Object class is a base class of all the other classes.

Module Descriptor

When we create a module, we include a descriptor file that defines several aspects of our new module:

• Name – the name of our module
• Dependencies – a list of other modules that this module depends on
• Public Packages – a list of all packages we want accessible from outside the module
• Services Offered – we can provide service implementations that can be consumed by other modules
• Services Consumed – allows the current module to be a consumer of a service
• Reflection Permissions – explicitly allows other classes to use reflection to access the private members of a package

Module Types

There are four types of modules in the new module system:

• System Modules – These are the modules listed when we run the list-modules command above. They include the Java SE and JDK modules.
• Application Modules – These modules are what we usually want to build when we decide to use Modules. They are named and defined in the compiled module-info.class file included in the assembled JAR.
• Automatic Modules – We can include unofficial modules by adding existing JAR files to the module path. The name of the module will be derived from the name of the JAR. Automatic modules will have full read access to every other module loaded by the path.
• Unnamed Module – When a class or JAR is loaded onto the classpath, but not the module path, it's automatically added to the unnamed module. It's a catch-all module to maintain backward compatibility with previously-written Java code.

We can see what these modules are by typing into the command line:

java --list-modules

Module Declarations

To set up a module, we need to put a special file at the root of our packages named module-info.java.

This file is known as the module descriptor and contains all of the data needed to build and use our new module.

Requires

Our first directive is requires. This module directive allows us to declare module dependencies:

Now, my.module has both a runtime and a compile-time dependency on module.name.

And all public types exported from a dependency are accessible by our module when we use this directive.

Requires Static

Sometimes we write code that references another module, but that users of our library will never want to use.

For instance, we might write a utility function that pretty-prints our internal state when another logging module is present. But, not every consumer of our library will want this functionality, and they don't want to include an extra logging library.

In these cases, we want to use an optional dependency. By using the requires static directive, we create a compile-time-only dependency:

Requires Transitive

We commonly work with libraries to make our lives easier.

But, we need to make sure that any module that brings in our code will also bring in these extra ‘transitive' dependencies or they won't work.

Luckily, we can use the requires transitive directive to force any downstream consumers also to read our required dependencies:

Now, when a developer requires my.module, they won't also have also to say requires module.name for our module to still work.

Exports

By default, a module doesn't expose any of its API to other modules. This strong encapsulation was one of the key motivators for creating the module system in the first place.

Our code is significantly more secure, but now we need to explicitly open our API up to the world if we want it to be usable.

We use the exports directive to expose all public members of the named package:

Now, when someone does requires my.module, they will have access to the public types in our com.my.package.name package, but not any other package.

Exports... To

We can use exports…to to open up our public classes to the world.

But, what if we don't want the entire world to access our API?

We can restrict which modules have access to our APIs using the exports…to the directive.

Similar to the exports directive, we declare a package as exported. But, we also list which modules we are allowing to import this package as a requires. Let's see what this looks like:

Uses

A service is an implementation of a specific interface or abstract class that can be consumed by other classes.

We designate the services our module consumes with the uses directive.

Note that the class name we use is either the interface or abstract class of the service, not the implementation class:

Provides With

A module can also be a service provider that other modules can consume.

The first part of the directive is the provides keyword. Here is where we put the interface or abstract class name.

Next, we have the with directive where we provide the implementation class name that either implements the interface or extends the abstract class.

Here's what it looks like put together:

Open

We mentioned earlier that encapsulation was a driving motivator for the design of this module system.

Before Java 9, it was possible to use reflection to examine every type and member in a package, even the private ones. Nothing was truly encapsulated, which can open up all kinds of problems for developers of the libraries.

Because Java 9 enforces strong encapsulation, we now have to explicitly grant permission for other modules to reflect on our classes.

If we want to continue to allow full reflection as older versions of Java did, we can simply open the entire module up:

Opens

If we need to allow reflection of private types, but we don't want all of our code exposed, we can use the opens directive to expose specific packages.

But remember, this will open the package up to the entire world, so make sure that is what you want:

Opens ... To

Okay, so reflection is great sometimes, but we still want as much security as we can get from encapsulation. We can selectively open our packages to a pre-approved list of modules, in this case, using the opens…to directive: