Overcoming the Organizational Challenges of IP over DWDM

On June 23, Light Reading’s Ray Le Maistre reported on the press release by Juniper and NSN in which the companies announced a parternship for delivering IP over DWDM (IPoDWDM) on Juniper line cards. In the article, Mr. Le Maistre cites a Heavy Reading finding that organizational challenges are  a substantial inhibitor of IPoDWDM deployment at the large carriers. This is completely accurate; however, the challenges can be overcome. Supported by a small group of very talented IP engineers, I’ve done it at a major carrier.


In many cases in the U.S., the largest ISPs have telephony roots. This was true in my situation. The support staff for Transport was accustomed to slow upgrade cycles and connection-oriented networks, neither being characteristics of IP. Organizationally, there were silos between the Transport engineers and the IP engineers.

Initial reactions from Transport on IPoDWDM was extremely negative. The idea of migrating transponders from DWDM terminals to router line cards, the foundation of IPoDWDM, is heretical to their traditional notions of how networks should be run.


In the past, Transport groups had to support a variety of voice, IP, and (what we now call) legacy data services. The services each had requirements that led Transport to develop a network suited for the lowest common denominator of packet loss, jitter, and latency needs. As IP emerged as the dominant service, the cost of over-engineering the physical layer became apparent to those who saw the price erosion in IP transit service.

Unfortunately, this message was a difficult one to pitch to Transport. The fear was that any relaxation of internal standards for DWDM networks would result in outages blamed on the Transport group. The group pointed to the DWDM vendor, claiming that any relaxation wouldn’t meet vendor standards. The vendor gingerly responded that it was the carrier that included certain specifications in RFPs during the early days of DWDM. Do you see the circularity?

Getting IPoDWDM Deployed

The deployment of IPoDWDM forces a unification between the groups if the effort is to be successful. Ideally, this means both Transport and IP departments report to a same VP, although other arrangements may work if management has a common view of what is needed to thrive in an environment of consistantly lower prices. I was fortunate in that the organization did the necessary alignment to recognize IP as a fundamental building block of all services.

In a IPoDWDM environment, Transport will cede much of the control it previously had on the DWDM network. Rather than using gray optics on the router, the router’s line cards will “color” the laser before sending it to the mux/demux of a DWDM terminal. Transport no longer has complete visibility into the signal. IPoDWDM sets the scene for the creation of a light path between two Layer 3 devices with minimal, inexpensive devices in the data path, a concept I first heard articulated by Peter Löthberg of Stupi AB.

Even with a combined organization, subject matter experts with a background in Transport sprang out of nowhere to deride IPoDWDM with claims that the “IP guys” have a cowboy mindset with regard to designing and operating networks. If you encounter this situation, don’t be outrightly dismissive. Concerns about polarization mode dispersion (PMD) over time and other related concerns often have some degree of validity.

Many of Transport’s concerns stem from a fundamental misunderstanding of IP and routing. Several items in this area come to mind.

  • IP networks are tolerant of packet loss– The idea that a 40 Gb/s connection can drop a few hundred packets a day without concern is unthinkable to Transport. The impact to the user experience is nil. Do you really want to invest in a regen over a few hundred errored packets? Their response – yes.
  • IP networks route around failure – TDM is not IP. The connectionless nature of IP allows routers to divert traffic from failed connections. Particularly if IP or MPLS fast re-route is deployed, the switch-over is nearly instantaneous.
  • 4 x 10 G does not equal 40G– A common argument against IPoDWDM was that the backbone could be enhanced with additional 10G circuits. This is specious reasoning, easily assailed by anyone with hands-on experience in the day-to-day operation of a large IP network. Setting IGP metrics to optimize routing in both steady-state and failure-state in a “spaghetti” network in almost impossible. Manual tweaking of metrics will be required upon failure. Anything manual in a large-scale operation equals higher mean-time-to-repair.

Much of what my team and I accomplished in implementing IPoDWDM was the result of adhering to basic change management concepts. I won’t delve into these, as you can read about the topic on the internet and in countless books.

If you find yourself in my position, consider these observations.

  • You are competing against Cogent and other operators that will leverage their lean cost structure to drive down price per Gb of IP transit service. If your company wants to build a gold-plated transport network, you will suffer in the marketplace.
  • Build your network for IP. If the DWDM transponder can be cost-effectively intregrated onto an IP routing device, what reason exists not to do it? Providers will not be building 40G/100G networks for TDM voice, ATM, and frame relay.
  • U.S. providers are following the lead of the European PTTs in outsourcing many aspects of operation and engineering of the network. Whether you not you believe this wise, a simple IP core with a minimal DWDM infrastructure will position your company to reduce support costs paid to the outsourcing party.

Deploying IPoDWDM is definitely not a simple exercise in an organization with an entrenched Transport group that views the technology as job-threatening (actually outsourcing is the true disruptor to the status quo, not IPoDWDM). With considerable patience and savvy, you will have the advantage in overcoming organizational challenges.

Jeff L.


Why I’ve Started Blogging

Welcome, readers. As a new entrant into the blogging space (sorry, I can’t bring myself to use “blogosphere”), I want to explain why I am doing so. For a long time, I considered the vast majority of blogs to be self-serving pinnacles of narcissism, akin to personal home pages in the mid to late 1990s (Yes, I had one).  I viewed the blogs–particularly the popular tech blogs–to be petty, sensationalistic, and dull. Of course, there was great content out there that I’d discover via Digg or other similar site. Still, I was not a regular reader of any blog, and I had no desire to start my own.


What’s changed my thinking? Since I started my business in January, I’ve received tremendous support from business associates, alumni, friends, and persons known only to me in the online realm. The wisdom and insight they provided is immeasurable. If, in my musings about IP, mobility, and entrenpreneurship, I can add something of value to the community, I would view my effort as time well-invested.