In 2003, I stopped being a systems software engineer and joined the Manageability team at NetApp. My then boss, Nawaf Bitar, had orchestrated a re-org to absorb that team under him and he saw this as an attractive opportunity for me. The team was very small and understaffed and very talented.

And I was an ambitious, obnoxious, 20-something determined to make my mark which was encouraged to go blow up the current piece of software architecture and build something new.

There were a lot of lessons I learned during that time. And many of them people lessons. And I’ll get to them in time.

But there was one that is particularly relevant to my day today, so I’ll repeat it here.

At the time, the storage management product was called “Data Fabric Manager.” The basic architecture and I am going from memory, was a monitor service that polled the infrastructure, an embedded database, eventing and alarm service that sent out SNMP traps, or emails based on what the monitor service uncovered.

The DFM CLI was a program that executed as a CGI-bin script inside of an Apache web server. The CLI implemented a web-UI and a CLI command set and had an XML input interface.

The problem with the technology was that in 2004, the kind of UI you could build in a web browser was quite limited.

At the time I believed that to build a slick performance monitoring tool, you wanted a thick-client and that a web-UI wasn’t going to cut it.

The team agreed to build a new thick client that would have a new API service, called Acropolis that the thick client would use.

Later on, we built Protection Manager on top of this new architecture, and Protection Manager required a lot of APIs.

And then there was a debate over whether the APIs would be public or not.

As we were building the UI, one of our most talented engineers observed he could be 5x more efficient if there were a private API that the UI engineers could use. His point is that the UI needed a lot of API’s and not all of these APIs were going to be useful to anyone but the UI.

And it was an interesting point. Here I was advocating for a public API at a cost to development at a time when no one was integrating into management systems.

And after some thought, I made the call that all APIs should be public.

And the rationale was the following

1. We had no idea what someone would use the APIs for
2. We had no idea when the APIs would be used
3. It’s practically impossible to justify investing in APIs except when your products need them.

And it became a mantra of mine when you create an API make it public. Because making the API later is very hard to prioritize and get resources for.

Yes, your API is probably not the world’s best API. But then again neither was MS-DOS or the x86 instruction set.

So what happened next…

Later on, when we needed to do integrations, integrations we didn’t anticipate that decision paid some dividends.

The existence of the APIs made it possible to have partnership discussions that centered around extending or improving the API instead of “whether APIs exist”. And because all of the functionality was exposed, the partner could play with the totality of the functionality even if we didn’t have everything they wanted. And more importantly, this allowed us to discuss how we could evolve the whole system to do the right thing.

 

 

 

 

My career has been a lot of fun. And as a career, I hope the best is yet to come.

There are three major pieces of technology that have shaped my view on software architecture.

The first was the delivery of a streaming media cache at NetApp. The streaming media cache was a feature of the NetCache product line. In this project, I was an individual contributor working with some of the best software architects I have had the good fortune of working with. What we built was a system that allowed us to sit between a windows media server or a quick time server or a Real Networks Server and delivery media at a lower cost per $ than having a bunch of servers.

The second was the set of products that I delivered as part of the storage management team at NetApp.  The first was a performance monitoring tool that created the first client-server product NetApp had called Performance Advisor. The second was a radical data protection tool that used policy-driven data management that was too far ahead of its time. And the third was the first solution to integrate storage and virtual machine management.

The third was my time at Zynga. What I did at Zynga, was to create a centralized operations team that delivered a shared back-end with a large of collection of services that reduced the OPEX and CAPEX of managing Web-scale infrastructures while simultaneously improving uptime. Some of the stuff we did was open sourced, like zperfmon and zbase. Our infrastructure and team were so amazing that after a mistake and a bug in a partner product, we were able to restore a 12 million DAU game with several thousand servers in less than 12 hours. The capstone of my time there was the delivery of an API infrastructure that took all of Zynga’s different backend services and put them behind a REST API that made it significantly easier to deliver features and services and enable 3rd party feature development.

Since then, I have had the good fortune to work at VMware, and a lot of what I am doing there I can’t talk about so will not. Although one thing that was accomplished as part of the 6.5 release, was an architectural review board that I chaired that reviewed many vCenter features.

The theme of my career, and what I view as strategic software architecture, is that I didn’t architect a single product. Most of the architects were people who I advised. What I did was to create the context that allowed them to deliver miracles. There were times, of course, where I had to step in, and advice requires technical understanding, but it wasn’t about me writing all or any of the code.

And this is where the confusion happens. I remember sitting in a room with a manager at NetApp. And she was screaming at me, telling me I was good for nothing worthless software architect. That a software architect created a working prototype like her husband. She was pissed that she was forced to work with me.

Given it was early in my career, I freaked out.

And I get her reaction a lot. Every time someone looks at me, and what I do they think – he doesn’t write code, so he can’t be any good. He just talks and talks and talks. And he doesn’t understand what’s really going on.

And then three years later they are like that woman who screamed at me. After we shipped the first version of the product, and more and more of the set of products started to adopt that unifying vision, she realized that I actually added a lot of value. That the whole org of over 200 people was moving in the same direction even though they were working on a variety of products. That what I did was create a context that enabled very large teams to work together well.

All that talking and probing and pushing and getting people aligned and providing the technical depth that allowed people to get unstuck produced a staggering amount of software, more than anyone human being could write in a year.

Because what I do is to figure out what are the strategic lever points, and use those lever points to move the world.

 

 

About a month ago, my wife and I attended a performance of Hamilton in San Francisco.

And afterward, as I was sitting in the car, I realized that there was a story I wanted to tell in defense of software architecture.

More precisely a story that was a  defense of the kind of software architecture I do. Because it’s a kind of software architecture that is extraordinarily valuable, and extremely undervalued: strategic software architecture. What strategic software architecture is and is not will be something this set of essays will attempt to define, describe, characterize and explain. In a nutshell, the central thesis is that for any business the strategic 5 -7-year question of how to marshall people and technology and product to deliver outsized business results is actually a software architecture problem that tries to impose structure and chaos on a fluid situation while providing flexibility in the choice of tactics.

A mouthful indeed.

And in honor of Alexander Hamilton and the Founding Fathers, I decided to write a series of Essays titled the architecturalist papers, a pompous homage to the Federalist papers, that tried to explain and defend strategic software architecture.

The struggle I faced in putting the finger to keyboard, was the daunting task of doing research. After all, shouldn’t I do some survey of the state of the, and show where my thoughts fit into the general understanding of software?

Thankfully, a friend of mine remarked that many researchers in her field are not empiricists. And first I had to look up the word empiricist and discovered that it meant

a person who supports the theory that all knowledge is based on experience derived from the senses.

And it became apparent, that I had a lot of experience in this space, and there was a lot of knowledge to be derived, and some more abstract thinkers could figure out general models that were more valuable.

What clinched the deal, was another exchange with my wife about common sense. Common sense, I had read somewhere was defined as the set of accumulated wisdom from experience. When we say someone lacks common sense, what we mean is they lack the accumulated experience or don’t have access to that experience and make poor decisions.

This set of essays is an attempt to share my collected set of experiences and will allow others and myself to derive knowledge from the entire experience and hopefully share some common sense ideas that have proven to be very useful over the years.

The next essay will be a survey of what things I have been involved in that forms the basis of my experience.