wrong tool

You are finite. Zathras is finite. This is wrong tool.

How Stanford Screws the Middle Class

by 3 Comments

One of my personal enduring mysteries was why do the top 100 private colleges charge about the same amount of money for tuition.

Given their wide variance in size, location, and endowments, you would expect to see a wide variance in price.

Except you don’t. The list price for a college education is about the same.

And then I spoke to someone who is deep in the bowels of Stanford’s budget and figured out how exactly Stanford is screwing the middle class.

Let’s begin with the following startling observation. Stanford has two sources of revenue. The first is a draw on their endowment. The second is their ability to issue bonds to borrow to build (check out http://bondholder-information.stanford.edu/home.html)  Tuition, is a drop in the proverbial budget, a rounding error.

Just to make it real, the draw on endowment is about 5% a year so

21.4 billion * 5% = 1.07 billion

Student tuition = 14k * 3 quarters * 7k = 294 million

Ah you say, look! it’s 30% of the budget… except. about 4679 get some kind of tuition reduction, so let’s cut that number in half so it’s about 150 million dollars or about 15% … A drop in the proverbial bucket in a billion dollar budget.

Let me think about this for a moment. Stanford benefits from tax exemptions from gifts and simultaneously benefits from tax benefits while borrowing money while demanding money it doesn’t need from parents after tax income.

Hmm…

Let me repeat, the tuition that basically destroys a college graduate’s ability to buy a house or devastates a parents retirement is a rounding error in Stanford’s budget and comes from your taxable income.

Is this about Stanford? Certainly not, it’s about Harvard and Yale and Brown and by implication every institution of higher learning that is charging more money because they can.

Why does your college education destroy your life and your parents retirement? Because we’re stupid enough to pay for it.

 

Apple v Google – c’est la guerre

by Leave a Comment

Just before Greece and Italy went to war, the Italian Ambassador came to General Metaxas and told him the terms for peace. Metaxas apparently replied in French that they both spoke: ‘Alors c’est la guerre.’

What had happened was that the irreconcilable differences between the two great nations created war. The goals of the Fascist Italian leadership and the national goals of Metaxas made war. And this in spite of the ideological alignment between the two leaders. Mussolini and Metaxas were both fascists.

Apple’s decision to create ad blockers was inevitable. Apple is primarily aligned with the folks who buy their devices. They are absolutely committed to building a compelling experience on their platform at the expense of everyone on their platform. Customers pay for the platform and Apple’s biggest source of revenue is that platform. That single-minded focus on the platform experience had to lead to ad-blocking because ads are the single biggest source of irritation in the web-ecosystem.

Google because it was aligned to maximizing ad revenues and not maximizing the web-ecosystem experience, left Google vulnerable and by extension the web vulnerable to Apple’s decision.

Google had about 15 years to make the web better, instead they focused on extracting even better revenues from the web through more efficient ad-revenues. Google never took ownership of the web-platform. It’s unclear that Google could have taken ownership or stewardship. Because Google didn’t own the platform.

This outcome, the  assault on the ad system that powers the web to the detriment of the user experience was inevitable. The only option is for Google and the ad networks it supports to figure out how to make ad’s better.

 

Packet re-ordering is bad.

by Leave a Comment

One of the weirdest things at Juniper was the obsession the networking teams had about reordering packets. They kept talking about how applications could not tolerate reordering.

And this confused me to no end.

After all TCP was based on the assumption of packets being reordered and out-of-sequence and surviving that mess?

And then it was explained to me as if I was the networking NOOB that I am. The problem is that when a packet gets reordered TCP doesn’t perform as well as when the packet gets sent in order. And there are scenarios where TCP will assume that the network is congested if the packet doesn’t get sent in time and will slow down the network connection.

And so to work around the TCP protocol thinking it understands what is going on in the network, ASIC engineers do heroics to ensure that packets flow through routers in order.

Then I read this today and I was reminded of those conversations:

http://gafferongames.com/2015/09/12/is-it-just-me-or-is-networking-really-hard/

There are all sorts of very interesting applications that run over the Internet that really are just pumping packets and want them arriving in order or not at all.

And that because of these applications the design complexity of routers is vastly more complex than if the layers above the network did not assume anything about a reordered packet.

 

Facebook hitting a billion people – is this the day that open messaging died?

by Leave a Comment

In the 1980’s visionary technologists created e-mail as an open non-proprietary messaging system. This allowed anyone to communicate with anyone on open networks.

With Facebook hitting a billion, and WhatsApp hitting 900 million people, we now have a new proprietary network that has the reach that email does.

In the open messaging world, messages were owned by the sender, and the recipient and were portable. Your social network – the set of people you interact with – and your chats were owned by the people who created them.

Facebook and Whatsapp have now up-ended that open communication channel. They own your social network and they own your messages and they have the reach to displace open communications. A private entity owns your friends and the relationship to your friends.

And hence snapchat and wickr. If there is no portability and durability that is independent of the service provider, then you may as well treat the messages as ephemeral.

Who would have thought, my most private and important data would be owned by someone else.

Oh brave new world!

The completely misunderstood IOPS

by Leave a Comment

I was recently in a meeting about performance where the discussion turned to how many IOPS was the database doing.

And what was interesting was how much of our thinking about performance is formed in a world where IOPS are a scarce resource because the underlying media was soooo slow.

In the modern, post spinning rust world, IOPS are practically free. The bottleneck is not the underlying media, SSD’s and later things like 3D Xpoint memory (what a horrible, horrible name for such an important technology) have essentially free IOPS. The bottleneck is no longer the media (disk drive) but instead the electronics that sit in front of the media.

The electronics include things like networks, memory busses, and CPU’s. We are now bandwidth and CPU constrained, no longer media constrained. What that means is – of course – interesting.

One practical consideration is that looking to optimize IOPS is no longer a worthy effort. Instead, we should be looking at CPU and Memory cost per IOP. And we should be willing to trade off some CPU and Memory for more IOPS to improve overall system behavior.

For folks, like myself, who grew up working really hard to try and avoid doing disk operations, embracing IO is going to be hard…

And like a buddy of mine once said, these material scientists keep investing these new exotic technologies that keep us system software engineers busy.

It’s a good time to work in systems.

Metrics over usability 

by Leave a Comment

  
This is the kind of shit that drove Zynga customers nuts.

In an attempt to drive metrics to other features … We add friction to the top activity… I didn’t know about collages and not do I care to know about them and I certainly don’t want to be reminded of them all of the time.

I used to be able to just enter a status, now I have to pick one.

This is just another example of egregious Facebook metric driven feature – like the hyper aggressive attempts to get me to turn on notifications.

The end of storage tiers

by Leave a Comment

I wrote about this in 2008 on my now defunct corporate blog at NetApp. It’s fun to be working at a company that can actually create the IOPS tier.

Flash has once again thrown into stark relief the absurd classification of storage into tiers

Talk to a storage vendor and Tier 1 is their most expensive stuff. Talk to a storage architect and Tier 1 is their most expensive stuff. If you’re lucky there is some overlap.

Then we have Flash. Is it Tier 0? Does Flash make Disk Tier 5? What is the role of Flash and Disk? Is Disk the new Tape? So do we need to have Tier -1 for storage that is faster than Flash?

Then there is the whole disk storage is secondary storage. Secondary to what?

I never really did get all of those classifications of storage into tiers. I tend to think of storage in terms of how it is used.

So instead let me propose a new model for storage tiers based on the ratio of application CPU and memory, the amount of IOPS required and the capacity needs of the application or the ratio CPU:memory:IOPS:Capacity.

Based on that ratio there are three storage tiers

  1. Captive IOPS, where IOPS are all dedicated to a single application. In this deployment the ratio is 1:1:1:1. Add more CPU and Memory and you add more IOPS and Capacity. Because of the nature of the application and how many IOPS it consumes, there is nothing left over for another application.
  2. Shared IOPS, where IOPS are shared across a collection of applications. In this deployment the ratio is M:N:1:1. As you add more CPU and memory, the number of IOPS increase but not at the same rate. So you can share the IOPS across a number of applications rather than dedicating them to a single application.
  3. Capacity Efficient where the number of IOPS is dwarfed by the capacity requirements. In this deployment the ration M:N:1:Q. where as M and N increase, Q increases but IOPS do not. A good example is a backup server. As more data gets backed you need more capacity, but you don’t actually need more IOPS. Another good example is a home directory where capacity needs increase, but actual IOPS do not.

Next, I’ll explore the implications of these three tiers.

The declining value of stock options in an IPO free world

by 1 Comment

One of the tricky bits about massive valuations, and long drawn out periods before an IPO is cash compensation for employees.

This article does a really good job of pointing out another emerging crisis.

The basic challenge is that as the time to IPO expands because of secondary private equity markets, employees begin to devalue the value of their options because they are not liquid putting upward pressure on their salaries.

Let me explain.

Suppose you earn 100k a year. A pretty awesome chunk of change mind you … Now at Big Corp you can expect to earn 400k over four years.

A startup makes an offer, and it has less cash and more equity. The cash component is 70k. And so at the startup you can expect to earn 280k over four years.

The gap between Big Corp and Startup is 120k over four years. Over eight years the gap is 240k.

Basically every four years at the Startup is equivalent to working one less year at Big Company.

Thank goodness you say for options. The options will cover the difference between the Startup and the Big Company, hopefully by a lot.

And here’s where things get dicey, the longer it takes to get the option money from Startup, the bigger the financial exposure from a startup failure.

In a normal environment to compensate for the increase risk exposure, you would need more equity. The longer it takes to see if an investment pans out, the higher the rate of return should be.

Let’s put it this way:

Big Corp Comp = 400k

Startup Comp = 280k  and 2 million dollar option outcome with 10% certainty.

The expected value of Big Corp is 400k

The expected value of Startup is 480k (280  + 2000*.10)

This looks pretty good! Except I am probably overstating the probability of the 2 million dollar option outcome… And if we start adding notions of present value of money, the startup outcome may not look so good…

If the time horizon gets extended out to 8 years,and option outcome is the same.

Big Corp Comp = 800k expected outcome

Startup Comp = 760k expected out come

Now you’re starting to lose out by staying at a startup.  And again if we consider that Big Corp gives you 30k a year more than Startup Comp, and that money can be invested then it’s even more favorable for Big Corp.

Therefore employees at Startups either have to larger salaries or bigger equity packages or have to believe that the option outcome is going to be much higher.

Given that the time horizon to liquidity events is increasing, my expectation is that salaries are going to increase.  If salaries do not then we may be in a bubble because we are all collectively underestimating the risk of the option outcome.

 

Android’s reach is the only thing keeping it relevant

by 2 Comments

Every so often some analyst publishes an article talking about changes in market share of iOS vs Android as a way to get people to read their articles. Like marketwatch just did. 

And those articles are just so wrong and simplistic in their analysis that I want to scream.

Let me help you read them a little bit better…

There are three ways you make money in the mobile market:

  •  Selling devices
  •  In app purchases / subscription services
  •  Selling advertising

There are two costs in the mobile market:

  • user acquisition (basically getting people to download the app)
  • development costs

And then there is the reality of income inequality  – there are a small number of people with a lot of money to spend on shit. 

Given all of this the most important question isn’t what is apple’s market share but:

of the people who have money how many does apple reach?

In this game, apple is dominating the competition and the competition is getting screwed in three ways:

(a) Apple’s lead in manufacturing is astonishing. 

(i) They are simply building shit that other people can’t 

When they build the Mac Pro with the sold aluminum case they bought all of the lathes that could make that case and two years worth of production of the lathes. That meant their competitors could only build a case like Apple’s 3 years after apple had already shipped the mac pro. 

(ii) Their supply chain is magical. Tim Cook, a supply chain dude, became CEO of Apple to reflect the importance of supply chain management. 

And this lead results in – better products – than their competitors can make

(b) Because of the share of people who have dollars, app developers are making better apps for iOS than android

If you have an android device, buy an ipHone and use the same apps, and you’ll see the difference immediately. The iOS are simply better built. And Apple has gone to great lengths to make it necessary to build two copies of the App. And given 5$ of investment, companies will go with 4$ on Apple and 1$ on Android. 

(c) Because of carrier subsidies

People are paying significantly less than list price for a phone and because carriers want to get people to use their networks, they will happily subsidize phones … This allows Apple to sell a high value product at a low price. 

Put differently:

Consumers always buy the best value, and the best value is the best built phone running the best apps at the best price and Apple has an incredible lead in both areas.  

Then why do people focus on reach?

Because I lied about the three ways.There is a fourth:

Use the phone to sell a service (Uber/Google Search/Facebook/Amazon) or enterprise customers like Microsoft Outlook/Prezi etc. 

For those vendors, having an app on the widest set of platforms is crucial to business success because it affects reach.

But because they have to be an all platforms, they have to port to every platform.

The real observation hear is that as long as Android has 80+% market share, for reasons of reach app developers will port to Android. But because the $$$ they expect to make, the ports will be crappier than the iOS versions. Maybe for some guys – like Facebook or Uber – they will have apps that are equivalently great – who knows. 

There is this theory  – Android has reach, and eventually it will make enough money. There are two problems with this theory:

Reaching the top 10% of income earners is more valuable than the remaining 90% because of income distribution. I would rather have 100% of the top 10% than 100% of the bottom 90% of global earners. 

Reaching 90% of the people is more expensive than reaching 10% of the people – in raw dollars. And so the value of each guy with less money is less than one guy with more money. 

If Android had no reach, we would be talking about anti-trust not about Apple’s demise. 

The real numbers to pay attention to with Apple is:

(1) What is their share of market share $$$ (and here it’s crushing the competition silly)

(2) Are enough apps making more money on Android that would cause the general App vendors to pivot their top resources to Android vs IOS

until (1) or (2) slip apple owns this baby reach ## be damned. 

Are Unicorns really dangerous to employees?

by Leave a Comment

One of the more intriguing questions, I keep asking myself, is whether the myopic fixation on bubble valuations is reasonable.

At the end of the day, I think what I have convinced myself is that

  1. investors are not investing at ridiculous valuations,
  2. founders and very early employees are cashing out, and yes
  3. some people are going to lose money
  4. it’s not like it was 2001 where retail investors buying on margin were buying theglobe.com

Then who cares? The reason I do is because

  1. I like to understand things
  2. In any new financial instrument someone is assuming more risk and someone less

And I am beginning to conclude that employees are assuming more risk.

In the valley, we’ve created a  system where you have a relatively low-base salary and a very high variable income. The goal of the system is to encourage employees to keep playing lottery with their time in the hopes that they strike it rich.

The problem with this system, from the perspective of the owners of the company, is that the more equity you hand over to the employees, the less equity the owners have.

If a company has 1000 shares, the goal of the owners is to maximize the number they own and minimize the number everyone else has. The problem is that the way employees think about their compensation forces the company to keep giving out shares to new employees, and thus the % of the company owned by owners shrinks over time. The company has to keep issuing new shares, because at some point they have no more shares to give out.  The owner to retain his share of the company, has to keep buying shares and that increases his risk as more and more of his money is put into one company. And one important class of owners, the founders, typically doesn’t have the cash necessary to preserve his share of the company.

Therefore, the goal of the owner is to minimize the number of shares issued for employees. An approach to solving that conundrum is to increase the per-share value. The way you increase per-share value is have investors buy into the company at a high valuation. The problem is that investors don’t want to assume that kind of risk for their investment. And so we have liquidation etc preferences that allow the valuation to be set high but the effective purchase price to be set low.

To keep dilution to a minimum, the founders are able to drive the value of the stock up with investor money and allow the investors to not assume the risk of the high valuation…

The risk of high per-share price is transferred entirely onto the employees for the benefit of the founder and early investors.

Brilliant…

Let’s try that again…

Any half-decent engineer will evaluate their salary like this:

Total Income = Cash + Equity

And stock option equity will be valued like this:

Equity = %of company (Expected Value at time of Cash Out – Current Value)

Any  RSU equity will bed value  like this:

Equity = #RSU * Current Value + #RSU * Expected Value of company when you cash out.

Suppose you are at a company X, your compensation at company X is TotalIncome(X).

When you go to a Unicorn what they will do to make a competitive and attractive offer is the following

TotalIncome(Unicorn) > TotalIncome(X)

Where TotalIncome(Unicorn) = Cash(Unicorn) + Equity(Unicorn)

The way they do it is by saying:

Equity(Unicorn) > Equity(X)

So far so good… Nothing wrong so far.

But remember the value of Equity is very dependent on two parameters:

  1. Current Value
  2. Cash out Value

And here’s where Unicorns can really hurt employees. Unicorns like to offer RSU’s.

  1. Because of the high current value of RSU they can offer a small number
  2. The cash out value – because it’s only common stock – only matters if the company IPO

Giving out a small number really matters to founders and investors who care about dilution. The more shares you give out, the less each share is worth. A high-growth company that is hiring a lot of people prior to the Unicorn phenomenon would keep printing shares to keep hiring employees and that would cause the early investors to get PO’ed. The stock dilution and the employee lockup was a big deal in 2001.

Not so much now.

And here’s how ….

If you are a Unicorn, any time you need to issue more shares or deal with compensation issues, you just artificially increase the value of your company through another round, and hand fewer higher value shares to new employees. This allows you to both simultaneously keep TotalCompensation competitive and keep the number of shares static.

If you are particularly craven as a Unicorn, you can have Cash be lower than your competition with a small number of RSU’s whose value is mythological.

so far so good.

And this is okay if and only if every single company IPO’s and the public markets agree to the private market valuations.

And that still would be okay if everyone was taking on the same risk. I mean everyone, founders, all employees and investors. Except they are not.

The founders benefit the most from the lack of dilution since they own the most shares at the lowest possible value.

The investors are buying into the company at a much lower value. Think about it, you as an employee are buying with your sweat equity valued at 1 billion, and the investor is buying equity valued at 200 million … The guy buying at 1 billion is going to be worse off than the guy buying at 200 million.

At some level, you can argue that I am just describing how start-ups work. And at some level I am…. Except …  the valuations are not being set in the public markets, but in private ones, and the valuation is being set to a billion for reasons other than the actual earnings of the company.

The Unicorn valuations are useful for retention, hiring, advertising, lead generation and ego and a product of a negotiation with little downside for the people doing the negotiation.

In effect, employees are getting less equity based on valuation that has nothing to do with the actual earnings of the company … Instead they are getting paid based on a valuation whose opacity exists by design.

Unfortunately, this kind of shit never ends well…