You are finite. Zathras is finite. This is wrong tool.
A few weeks ago I asked – what exactly happens when your employer treats your personal data as unimportant?
Why you get sued.
I am delighted.
Only when the cost of mishandling private data is high will the average business actually try and secure our private data.
A few weeks ago I called for a Hippocratic Oath for data scientists.
Now the New York Times is calling for regulation.
We can fight regulation all we want and we will.
But…
Customers will fight us. And they will fight us by moving to vendors that offer privacy and encryption. Where data is wholly owned by the customer.
And the customer will win. Because if we don’t act like their data is a sacred trust they will learn to distrust anything that they cannot control.
And they will control it through encryption.
And then we will have the medical industry where breakthroughs are there if only we could look at data …
After reading about the attack on Sony Entertainment, I am thinking that instead of worrying about hackers stealing from your bank we should be worried about hackers stealing from major corporations.
And that isn’t really surprising.
If identity is the seed of theft, then as banks and retailers get marginally better the soft squishy underbelly of HR departments becomes an inviting and easier target.
To date, a company cares about their business secrets being stolen, followed by their customer data being stolen and ending somewhere below not-at-all their employee’s private data being stolen. Employees don’t quit their job if their employer loses their data… At least not yet.
Identities are simply too valuable an asset for the bad guys to not want to steal, and unlike retailers where you can choose to not give them something that’s worth stealing your employer must have everything…
Me-thinks there is an opportunity for an innovation where companies can use tokens from some really secure system instead of the real raw data that they are just simply incapable of securing.
This Sony attack isn’t the end it’s the beginning.
Yesterday I wrote about how the failure of companies to respect the privacy and happiness of their customers posed an existential threat to the entirety of services that relied on big data.
Some folks on twitter remarked that my Data Scientist Hippocratic Oath is what their companies live and breathe.
@mattocko@kostadis_tech at LinkedIn statements like that oath were part of the company values, which were drilled into you every day.
— Peter Skomoroch (@peteskomoroch) November 20, 2014
And that’s great. I think that protecting user-data aligns with being a great company… And I think a great company sometimes may need to be explicit about how it thinks about user data.
Juliet asked how does this apply when the customer isn’t a person? I guess we need to refine the oath to be a little bit more specific – instead of customers we should talk about people.
And while I called out uber and facebook in my post, it’s only fair to share with folks that Facebook has been working to create a better code for it’s data science efforts described here and that the folks at Uber have hired an outside team to look at their approach to privacy.
I believe the self-interest of companies with vast amounts of data that we want to be kept private will ensure that the data is private because if it isn’t we will have encryption deployed everywhere. And I am delighted to see that happening.
And we’re seeing that. We, as users, need to demand that kind of data protection. The alternative is what happens in medicine where data is so regulated that our ability to fight diseases is being impaired.
Let’s talk a little bit more about Firewalls and Network Databases.
The obvious insight most networking people have is that the purpose of IP is to find a route between A and B. If there is a path, IP will find it.
The problem is that the network is a graph that isn’t owned by anyone. Anyone can create a connection, and any connection can create a path and any path can lead A to B.
The really annoying problem is that because the graph is built on physical cables that are very expensive to put in the ground, a single physical cable can be a path that is open to some packets and closed to others.
In other words, some links from A to B are valid because the person at A is allowed to talk to B, and some are not. And the amount of state you need to inspect to determine if packet A is allowed to go from A to B is considerable.
Really obvious stuff.
If you want to control traffic going that goes through a physical cable then you have to put a device that can look at all of the traffic coming in on the physical cable and make routing decisions. The device has to have a notion of forwarding paths and rules about how the data can flow through the device.
And that’s what a firewall does. First the firewall makes sure the packet is allowed to go through the firewall at all, then the firewall looks at where the packet wants to go next, and sees if the packet is allowed to based on the security policy.
From a particular point of view a firewall is just a router that uses a different kind network database. However, the database properties of the firewall are very different.
What is very confusing, especially to the networking industry, is that because firewalls are really just a router that has a different kind of database, it’s easy to assume that a firewall is a router and that the same kind of system level insights that apply to routers apply to firewalls.
And that’s unfortunate. Because a firewall system design is better informed from the system design of scalable web applications that have centralized databases than routers or switches.
More to come.
Following up to yesterday’s post, I wanted to talk about the Network…
Again this is stuff a noob like me finds interesting. I am glossing over massive amounts of detail. And I may be wrong, and if I am I’d like to be corrected.
If you come from a database background, a database has the following useful properties:
In an IP network, the database is the network, and the database has the following perverse properties:
The Network Database is the single most unique database on the planet. The properties of that database provide for resiliency, force co-operation between bitter commercial rivals and require some of the most sophisticated protocol engineering on the planet.
And why is this important to Firewalls?
If we consider firewalls, and computationally cheap security then we realize that the single most important function of a firewall is to make sure that Network is a Database only contains records (routes) that are approved.
That was a mouthful.
Consider a route from IP 1.1.1.1 to 2.2.2.2. The network administrator would like to say that no such route must exist in the network database. The problem is that the network administrator can not guarantee that such a route won’t get created either maliciously or intentionally. The inherent properties of the Network Database make this impossible.
What to do?
Well you create a device that has all of the rules that you want to enforce and that device has all of the properties of a real database:
And then you put that database in the middle of the network and have it verify that no invalid route is inserted by essentially seeing every packet and making sure that every route is valid in the context of the security policy.
In effect, this is the really astonishingly surprising result – a firewall’s first and foremost role in a network is to secure the routing tables. What I mean by securing the routing tables, what I mean is that you are ensuring that no route that you don’t want exists.
That’s why a firewall is inline, that’s why it exists. If you could secure the routing tables without a firewall, then you could – in principle – transform the security industry.
And it turns out that SDN can eliminate the need for a firewall… Maybe. And that is very disruptive. And SDN’s might do this because, well, they move the Network Database out of the Network into a database that has standard database properties…
And I’ll get to that when I talk about SDN.
Following up from my post about an Idealized Firewall , let’s look at what a Firewall does …
Again this isn’t novel or new…
First a picture:
A firewall provides three kinds of security services.
The most important service a firewall provides and perhaps the entire raison d’etra is what I call cheap security. Not cheap from a cost, but cheap computationally. The amount of state that has to be kept and tracked is relatively small. This kind of security is what we traditionally call Layer 4 security. A lot of what passes off as security in this space is handled both in a firewall, and using other technologies like vlan’s, ACL’s etc.
This is the layer that transforms packets into flows ..
Everything else a Firewall does stems from this layer, and I’ll get to why in another post.
Remove the need for this kind of security as a stand alone box and you remove the need for firewalls.
Because a firewall gets deployed at a network choke point, and because it does cheap computational security and is already inspecting traffic flows or must inspect traffic flows, a firewall is a natural place to deploy some connectivity services like NAT or ipsec. You can, and Juniper does, deploy this technology elsewhere like routers.
This technology only gets deployed on a firewall if the box is deployed at the network choke-point.
As you get better at solving the cheap security problems, the bad guys have to expend more effort to break in. As the efforts to break in costs more in terms of CPU and memory, then the amount of resources required to detect the bad guys increases. Like in a real war, as the weapons of defense improve, the offensive strategies adapt and force the defensive strategies to get more expensive.
Firewalls are able to play in this space because they do the heavy lifting of transforming the packet flood into a flow, and this layer is really just doing deeper and deeper analysis into flows and across flows.
Many innovative startup companies are working on this space because this is where the need is most pressing and most difficult to deal with.
If another layer were to transform the packets into flows, then this layer could live outside of the firewall. And in fact, as some companies have demonstrated it is possible to do this packet-to-flow transformation without being a firewall by hiding behind SPAN ports.
Because of the need for rapid innovation, resource requirements and lack of universality computationally expensive security can exist for a long time outside of the firewall before collapsing to the firewall.
When I joined Juniper, my interview was very funny because I knew nothing about networking or security. Well okay, I could draw clouds that had arrows pointing to each other and I knew the basics of sockets, but nothing like anyone who actually knows networking…
That’s what has made the job so much fun. Applying 15 years worth of experience to learning a new problem domain…
A specific challenge, I had was how to think about a firewall, in particular what exactly are the functional elements of a firewall that transcend any specific implementation.
A diagram I came up with that has had surprising predictive power is what I am sharing today.
Basically a firewall has three layers.
Starting from the top, we have a physical layer that has packets come in as a rain drops. I’ve already talked about rain drops and streams, and this lowest layer is what handles the packets. This is a physical network layer where cables are plugged into a firewall device. Just above the packet flood is a load-balancer that ensures that packets from similar flows end up on the next layer at the correct processing unit.
Cheap computational security is what I have been calling security that can be done very efficiently, has to look at a relatively bounded number of packets and can be assisted with hardware. This is typically what we have historically called “L4 Firewall Security”. Things like ALGs etc are implemented in this layer. This layer is able to take the packet flood and turn it into a stream before the security processing begins.
Because cheap computational security is implemented in terms of flows, a series of packets turns into a flow and then we start doing security on it, abstractly we can think of thousands of individual threads each handling a single flow. Obviously this is not how we would implement things, but is a useful mental model.
This mental model is particularly useful because it explains why the prior layer must load-balance the packets. Essentially using packet only information the physical layer must direct packets to the right thread processing the right flow.
Because the physical layer can make a mistake in load-balancing, this layer is able to detect the mistake and forward the packet to the right cheap security element.
Once the cheap computational security is done, then packet is forwarded to more expensive computational security if that is appropriate. That forwarding goes through yet another load-balancer whose purpose is to find either a lightly loaded computational element or send more packets to one that is still processing.
The final layer in firewalls is expensive computational security. This is stuff we have historically called “L4-L7 security services” such as UTM or SSL forward proxy etc.
This layer expects to receive a stream of packets and typically takes a lot of CPU and Memory resources to perform the operations that need to be done to detect security breaches.
Again this layer can conceptually be thought of a series of distinct computations on independent flows.
What I described isn’t a firewall per-se, it’s just a useful mental model for how to think about firewalls. And in particular, to understand the hardware trade-offs in designing firewalls.
One of the funnest parts about having a job in a industry you know nothing about is that you can ask really basic questions. Those questions allow you to build mental models that you can then use to learn about an industry and perhaps if you are extraordinarily lucky and pick the right time and place make an impact on an industry.
What follows are really basic insights, nothing profound here…
When I looked at Firewalls, the networking part of the firewall was very mysterious. Most of the value was in security, but a lot of the hard-back-breaking work was in integration with the network. Especially at scale the network integration was extraordinarily important.
At first I thought it was about protocols. And there is some truth to that but a lot less than we all think. The need to integrate in line with thousands of protocols is absolutely a challenge and a barrier to entry but not the real source of the challenge.
Maybe a different way to express my comment is that the protocol integration challenge is a signal that there is something going on there that is causing complexity, protocol integration is a symptom not the cause.
And my latest way to express the problem is the following:
If you imagine an IP network it’s a flood of packets like raindrops falling from the sky. And at every hop, decisions are made on a per-packet basis.
But security can not be implemented on a per-packet basis. You have to look at a large number of variable packets. The more complex the security system, the more packets that need to be looked at.
The challenge with firewalls is that you need to turn the flood of packets from millions of Alice’s, into little streams of packets that you can then inspect and once done inspecting flip that stream around into packets that the millions of Bob’s are expecting and send them out. Later on you have to be able to get packets from a specific Bob and correlate that packet with packets you received from a specific Alice talking to that specific Bob , turn Bob’s packets into a stream, inspect the packets and send the packets back out to the right Alice.
And – oh by the way – Alice and Bob must never know that you did that because the protocols may not allow that.
I have this image of water drops coming in one end, you enter the firewall and they turn into streams, and then they leave the firewall as raindrops the other side. And this image is remarkably resilient to explaining many things about how firewalls work and why some things in firewalls are easy and somethings are very hard.
And the nice thing about this model is that it explains why protocol processing is so crucial … without the ability to process a protocol it’s impossible to make the rain drop to stream to rain drop transformation correctly and because you are looking at all of the rain drops trying to find a bad guy, protocol completeness is crucial – skip a protocol and that is the door way for the bad guys.
In Episode IV of Star Wars, Han Solo, Luke Skywalker, Obi-Wan, and Chewbacca are trapped on the Death Star after their jump from hyperspace.
The Storm Troopers are quickly overwhelmed, and our heroes can access a physical terminal. R2D2 is then able to plug into the computer systems of the Death Star.
R2D2 can quickly access all of the information, including schematics and prisoner information.
In the post-mortem on Coruscant, I can imagine the dialogue:
Palpatine: How were they able to access all of the information?
CISO for the fleet: Well, the Grand Moff decided that the cost of adding firewalls and security systems to partition the network was too costly. He chose to rely on a big ass external firewall. His priority was the ability for his teams to access the information not to protect it.
Palpatine: A single droid was able to quickly and trivially get all of our operational information … Because we had no firewall?
CISO for the fleet: visibly sweating Well it’s more complicated than that. A firewall would have delayed the attack, and at the very least made it harder but nothing could have protected us against a determined attack.
Palpatine: A single bot that was put behind our firewall was able to get everything…
CISO for the fleet: Grand Moff Tarkin felt that it was impossible for a bot to escape the station or communicate externally…
Palpatine: Grand Moff is dead?
CISO for the fleet: Yes, Grand Moff is dead.
Palpatine: Pity. At least we won’t need to replace the commander of our space station. I suppose we’ll need a new CISO for the fleet.
Blue lightning crackles from the Emperor’s hand. The CISO for the fleet crumbles. His second in command steps forward…
New CISO for the fleet: Emperor, we’ll re-organize our security protocols immediately.
At Zynga, our security team was – actually – ahead of the curve. Our strategy was not to rely just on a hard shell. We also created internal segmentation of our systems. Basically, we created firewalls around each of our games and each of our systems. This kind of internal segmentation was a layer of protection that I thought was standard practice. More honestly, I thought this kind of protection was unnecessary. The recent disasters show that it is not. Too many people rely on a single external hard shell … unfortunately, once you get through the hard shell, everything is available.
This kind of internal segmentation is not as yet standard practice across the industry, nor was it standard in a galaxy far, far away…
And in all cases, the results were not that pretty…
