One thing I do not understand is how enterprise companies will rely on infrastructure they cannot restore without help from the vendor from a backup.
My belief is that a system exists only if you, without the vendor, can restore it from a backup.
A system that cannot be restored from backup doesn’t exist.
It is contingent on other forces that can destroy it at any point in time.
And that, as a result, relying on that system for anything that must survive the contingent force is irresponsible.
So, for example, if your business relies on a system that cannot be restored from backup and that system lives in a data center, then you are saying that your business is contingent on the availability of that data center.
Some will argue that, well, you could always restore things if you have enough time, and my answer is yes, you can if you have enough time, but what is enough time?
If the system is complex enough, time can be months.
And if the time when it needs to be restored is measured in days, then the fact that it could be restored in months is irrelevant.
Recently, I read about a man whose son locked his entire Google account. As a result, all of his emails, his contacts, and his contracts were gone. And with taxes due in a few weeks, he could not file taxes, he could not reach out to customers, and he could do nothing. Bills could not be paid. Why? Because he had no backup. He had critical data that was owned by Google. Not by him. And when Google decided he no longer had access to it, it was lost.
That anyone would allow themselves to be in that situation is a mystery to me.
When I joined VMware, I discovered how we tested back in 6.0. We created a new vCenter, backed it up, restored it, and declared success. I asked the team to take a backup of a system that was running, and try a restore, and guess what, the restore didn’t work.
It struck me as mind-boggling that with that VCenter in 2015, it could not be effectively restored from a backup.
Over the years, I struggled to make infrastructure, in particular, VCF, something that could be restored from backup, but what I discovered is that nobody cared.
I don’t mean nobody at VMware cared. I mean, the customer base didn’t care.
This utterly confused me. I could not believe it.
It was only when I went to a customer that I realized that the customer didn’t trust the backup. It trusted VMware to do whatever it took to restore a system.
And what I realized was that the constraint on the customer was how many engineers VMware had on staff to recover backups when the backup procedures failed utterly.
In many ways, the VMware engineering team was insurance for the entire industry should customer systems fail spectacularly.
If you depend on VMware engineering, you are at the whims of whoever runs it and whether their interests are aligned. And to be 10000% fair, this is true of any vendor on the planet.
Broadcom, by stripping the engineering team of redundancy, created a scenario in which a catastrophic failure requiring a large surplus of engineers would result in extraordinarily bad outcomes for the world as a whole.
Why? Because the only backup that any customer can rely on effectively is the VMware engineering team, and that VMware engineering team is smaller.
What has happened is that the industry has said, “ I don’t need to have a backup because I can trust VMware.”
But you can only trust VMware as long as VMware’s interests and yours are aligned.
And when that is no longer the case, the fact that you don’t have backups that you can restore from without VMware means that you are at the mercy of VMware’s business priorities.
And this isn’t about VMware; this is about any company.
Backups you can restore from are your insurance policy if the Vendor fails you. If you can’t prove you can restore, your single point of failure is another business that can change on a dime.
And for me, as someone who had to restore a business from a backup in 12 hours or risk an IPO, the idea that you wouldn’t have your own backups and rely on somebody else is unbelievable and unfathomable.
Leave a Reply