31 Jul 2008 17:29

Public and private entities can use deep packet inspection to analyse internet users' traffic, with potentially serious ramifications for privacy and the nature of the web

Anyone who uses the internet needs to be aware of deep packet inspection, its uses and potential misuses.

You may recognise deep packet inspection (DPI) as something internet service providers (ISPs) use to conform to the Communications Assistance for Law Enforcement Act (Calea), the US government-ordered internet wire-tapping directive. If that's not enough, DPI, albeit behind the scenes, allows ISPs to block, shape, and prioritise traffic, which is now fuelling the net-neutrality-versus-traffic-priority debate. So, what is DPI and how does it work?

Deep packet inspection
DPI is next-generation technology that's capable of inspecting every byte of every packet that passes through the DPI device. That means packet headers, types of applications and actual packet content.

Up until now, this wasn't possible with intrusion-detection or intrusion-prevention systems (IDS/IPS) or stateful firewalls. The difference is that DPI has the ability to inspect traffic at layers 2 through to 7 — hence the 'deep' in DPI.

A simple analogy would be that of snail mail. IDS/IPS firewalls would be the mail sorters who just read the letter's address, knowing nothing about the letter's content. Inspecting internet traffic from layers 2 through to 7 would correspond to the person who actually reads the letter and understands the contents.

To recap, DPI allows the people controlling the device to know everything, including the payload of each packet in the data stream. For example, if an unencrypted email is scanned, the actual body of the email can be reassembled and read.

Nate Anderson wrote an excellent Ars Technica article, Deep packet inspection meets net neutrality, Calea, in which the following quote appears:

"Deep packet inspection refers to the fact that these boxes don't simply look at the header information as packets pass through them. Rather, they move beyond the IP and TCP header information to look at the payload of the packet. The goal is to identify the applications being used on the network, but some of these devices can go much further; those from a company propecia baldness hair loss like Narus, for instance, can look inside all traffic from a specific IP address, pick out the HTTP traffic, then drill even further down to capture only traffic headed to and from Gmail, and can even reassemble emails as they are typed out by the user."

Anderson also explained what happens at layer 7:

"Layer 7 is the application layer, the actual messages sent across the internet by programs like Firefox or Skype or Azureus. By stripping off the headers, deep-packet-inspection devices can use the resulting payload to identify the program or service being used. Procera, for instance, claims to detect more than 300 application protocol signatures, including BitTorrent, HTTP, FTP, SMTP and SSH. Ellacoya reps tell Ars that their boxes can look deeper than the protocol, identifying particular HTTP traffic generated by YouTube and Flickr, for instance. Of course, the identification of these protocols can be used to generate traffic-shaping rules or restrictions."

What makes DPI all the more impressive is that the packet analysis happens in real-time, with data stream throughput approaching 20-30Gbps. With no loss of throughput, ISPs are able to insert these devices directly in their data streams, forcing all traffic to pass through the devices. Procera, Narus, and Ellacoya are front-runners in the development of this technology, having placed equipment throughout the world.

DPI's potential uses
DPI technology is unique in that, as of now, it's the only way to accomplish certain US governmental security directives. DPI also has the potential to do a great deal of good. For example, distributed denial-of-service (DDoS) attacks are virtually impossible to thwart. Conceivably, if DPI were in place and configured correctly, it would detect the DDoS packets and filter them out. Some more potential uses are listed below:

• Network security: DPI's ability to inspect data streams at such a granular level may prevent viruses and spyware from either gaining entrance to a network or leaving it
• Network access: DPI creates conditions where network-access rules are easy to enforce due to the deep inspection of packets
• Calea compliance: DPI technology augments traffic-access-points technology used initially for governmental surveillance equipment
• Enforcement of service-level agreements: ISPs can use DPI to ensure that their acceptable-use policy is enforced. For example, DPI can locate illegal content or abnormal bandwidth usage
• Quality of service: P2P traffic gives ISPs a great deal of trouble. DPI would allow the ISP to instigate traffic control and bandwidth allocation
• Tailored service: DPI allows ISPs to create different services plans, which means users would pay for a certain amount of bandwidth and traffic priority. This point is controversial and affects net neutrality
• DRM enforcement: DPI has the ability to filter traffic to remove copyrighted material. There's immense pressure from the music and film industries to make ISPs responsible for curtailing illegal distribution of copyrighted material

The above applications have the potential to give users a better internet experience. Yet it wouldn't take much mission creep to create major privacy concerns. It would be remiss if these were not pointed out so that everyone can understand the ramifications.

Possible misuses of DPI
DPI is another innovative technology that has ISPs arguing with privacy advocates. ISPs and DPI developers are adamant that the technology is benign and will create a better internet experience. However, privacy groups have two major concerns: that there would be little or no oversight, and the potential for losing still more individual privacy. Many experts find the following uses of DPI to be especially troubling:

• Traffic shaping: Traffic shaping is where certain traffic or entities get priority and a predetermined amount of bandwidth. With the increasing number of bandwidth-hungry applications, ISPs are having to make decisions on whether to increase available bandwidth with infrastructure build-out or increase control of the existing bandwidth. Installing a DPI system is usually the choice, as it's cheaper and has a more predictable return on investment. Albeit cheaper, it's riskier, and that may be why the net-neutrality debate is going on at the moment 
• Behavioural targeting: Behavioural targeting uses DPI technology for the sole purpose of harvesting user information anonymously — supposedly — and selling it to interested parties who use the information to create ads that are targeted to the individual

Final thoughts
This is a very complex subject, with the potential to change everyone's view of the internet. An optimist would say that DPI will help enhance the experience, even producing ads that are relevant to each individual user. However, a pessimist may say it's Big-Brother technology that only benefits ISPs. No-one is sure how the internet will look when the dust settles around the issue of DPI, but it should be interesting.

Michael Kassner is a network field engineer and independent wireless consultant.