The VCS high availability daemon (HAD) runs on each system.
Also known as the VCS engine, HAD is responsible for the following functions:
Builds the running cluster configuration from the configuration files
Distributes the information when new nodes join the cluster
Responds to operator input
Takes corrective action when something fails.
The engine uses agents to monitor and manage resources. It collects information about resource states from the agents on the local system and forwards it to all cluster members.
The local engine also receives information from the other cluster members to update its view of the cluster. HAD operates as a replicated state machine (RSM). The engine that runs on each node has a completely synchronized view of the resource status on each node. Each instance of HAD follows the same code path for corrective action, as required.
The RSM is maintained through the use of a purpose-built communications package. The communications package consists of the protocols Low Latency Transport (LLT) and Group Membership Services and Atomic Broadcast (GAB).
The hashadow process monitors HAD and restarts it when required.