AHV prioritizes system consistency over workloads, whereas ESX and every other OS prioritizes workloads over system consistency.
If you examine the fundamental difference between AHV and ESX, once you set aside the features, APIs, and opinions, the most basic question is: “When is the host down?”
ESX asserts that as long as the kernel is running, the host remains up because a workload may be running or about to start. Even if the kernel is unreachable from the outside, ESX continues to run. The only person who can decide the host is down, therefore, is a human.
AHV, on the other hand, believes that once it is no longer part of the quorum, the host is down.
Both approaches have value, but they yield different outcomes.
With ESX, the human has complete control over deciding when to restart a host. Because only the human knows whether the host is running, each additional component must continue functioning until instructed otherwise and must keep operating even if other system parts are down.
It’s why, for example, with vSphere HA, even if the network is partitioned, all hosts will run workloads.
Until the human indicates that ESX is down, all system components should assume a workload is either running on the ESX host or may start running, so they must try to keep running as well.
The difficulty is that a malfunctioning piece of software can appear just like one that is very slow.
Therefore, each layer advances without knowing if another will do the same later, which can result in incorrect decisions.
A trivial way to prove this is with backup and restore. When you restore a system from a backup, to the outside observer, that’s indistinguishable from a very slow system. The ancient system must now catch up with the current state of the world. To do that, it has to be able to read the current state, but there is no precise current state. So at some point, a human must be involved to resolve inconsistencies. It’s why restoring VCF is so painful.
The benefit of this approach is that it allows surprisingly fast delivery of components, as long as integration and consistency are less critical than the speed of feature delivery for each one.
The downside is that when two systems need to agree on the system’s status, they cannot. Because only a human knows if the system is up, down, or slow, any software trying to coordinate between two components can only make an educated guess about what’s happening.
To handle this, you need to invest in more tools, monitoring, and observability. But it’s always a guess.
The alternative approach of AHV has the property that the software systems are aware when the host is down, since the computer makes the decision independently of whether workloads are running on the AHV host.
More importantly, any workload on that AHV host will not run until the cluster control plane reinstalls it on the host.
As a result, any layered system knows exactly when to stop.
Consequently, all layered systems are aware of each other’s state.
And all parts of the system agree on the state of the workloads.
The upside of this approach is that the system is correct, scales better, and is simpler to operate and develop against. The downside is that until the quorum system is more reliable than a single kernel, your system is less reliable.
What AHV has done is make its clustered system as reliable as a single kernel. And that is an astonishing achievement.
Once that is achieved, and if overall system behavior is more important than any one single system, then the simplicity of the AHV approach allows for faster feature delivery because the complexity of integration is simple.
