Cluster formation
When distributed caching is enabled, a distributed cache cluster is formed in a shared environment among multiple Access Decision Service (ADS) instances, also called nodes. The process consists of discovering available nodes within the environment that form the cluster, and then establishing a robust communication between them.
Cluster discovery methods
ADS supports the following two different discovery methods for identifying cluster members within your network:
- IP Multicast
- DNS Query
Cluster name is the key factor for cluster formation. Regardless of the discovery method, a cluster will only be formed among nodes with the same cluster name.
IP Multicast
IP Multicast is a communication method for simultaneously transmitting data to multiple recipients and relies on User Datagram Protocol (UDP) to send messages to all nodes within a network.
By default, the distributed cache feature uses multicast to identify cluster members and send multicast messages to all nodes within the network. However, it's important to be aware of possible network overhead and flooding, especially when working with a large number of nodes in the network.
While multicast can function within a Kubernetes environment, its efficiency may be less than optimal due to network configurations and limitations imposed by Kubernetes.
DNS Query
Distributed cache can also use DNS (Domain Name System) as the method of network discovery and identification of nodes and cluster members within a network. This method involves querying DNS Address or DNS Service records (SRV records) with an external DNS service in order to discover cluster members.
In Kubernetes environments, DNS records are automatically generated for services. More specifically, in ADS deployments using Kubernetes, the Helm chart provided by Axiomatics generates a headless service, which provides information about all active Axiomatics service members in the cluster.
By using DNS-based discovery, you can simplify the management and accessibility of nodes and cluster members of your network. DNS also allows for dynamic updates, enabling nodes to register themselves with the DNS server when they join the network or cluster. This automatic registration and resolution of node names streamline the process of adding or removing nodes in dynamic network environments.
Cluster communication
Once the cluster is established, nodes use TCP (Transmission Control Protocol) for intra-cluster communication. You can also increase the security of the communication within the cluster by using encryption, as described in the Encrypt cluster communication section.
Depending on the number of nodes in the cluster, the process of cluster discovery and replication may cause overhead and increase the network load to your system.
Entry expiry and consistency
When an entry in the distributed cache expires on a node, a mechanism is in place to maintain a consistent state across all nodes. Specifically, the expiring node sends a remove command to other nodes, ensuring that the cache remains in a synchronized and coherent state. This approach helps avoid stale data and ensures that all nodes have consistent information.
What's next?
Deploy the distributed cache feature in the deployment configuration file according to your specific solution and optionally encrypt communication between the nodes of your cluster.