Internet Protocol and Optical Networking

As if in recognition of the turn of the century, the volume of data traffic finally surpassed that of voice in the year 2000. Forecasts over the last 15 years had often predicted this to happen sooner. What made it happen was the Internet and the revolutionary applications that it enabled. The Internet and all its applications use Internet Protocol (IP) packets, making IP the now-dominant form of all traffic. With the growth rate of Internet traffic between 100% to 200% per annum, and 3 to 5% for voice, IP data will only continue to dominate. As voice and other TDM service connections become a smaller fraction of the total traffic, they too will be converted to IP packets for switching and transport. These developments—and parallel advances in DWDM for optical networking—lead to an "IP over optics" paradigm that corresponds directly to the view of "services over transport" in Chapter 1. This is important background for transport network planning as it defines the technological framework for most architecture and design problems in the future. It also introduces new aspects to network design and operational problems, such as wavelength conversion, service-dependent survivability requirements, dynamic transport demands, and the need to plan in a way that accommodates uncertainty about future demand patterns.

Existing transport networks are not particularly efficient at handling huge amounts of data traffic or on-the-fly provisioning. Worse still from a network operator's standpoint, revenues from data are low compared to those from voice, especially given the investments the operator has to make to keep up with data growth. Operators are looking for ways to support data transport with less equipment and with more flexibility and provisioning speed than offered by a full stack of IP, ATM, SONET and DWDM layers. They also question the need to have ubiquitous SONET-layer ring protection, that at least halves the use of installed capacity. Each existing layer is, however, designed for particular functions. IP is best for end-user applications and the huge base of existing Ethernet LAN traffic. But native IP networking lacks reliability. ATM is good on quality of service and reliability, but is not widely used in the end-user environment and is relatively intensive in terms of cell processing and overhead. SONET provides low-delay, low error-rate transport, with protection ring mechanisms defined, but at the same time is rigidly channelized for TDM applications. DWDM provides a way to multiply the physical layer fiber infrastructure to cope with the growth, but lightpaths are expensive and need to be used flexibly and efficiently. Thus the conventional situation for IP data transport is a stacking of usually four layers:

IP— for user applications and LAN environments

ATM— for virtual circuit / virtual path capacity engineering, flow control, performance monitoring, virtual networking and QoS guarantees in the data networking layer

SONET— for high quality transport of payload over the fiber physical medium, error monitoring, OA&M, TDM synchronization and protection switching

DWDM— for sheer capacity, effectively multiplying the number of fibers in the ground

A problem with so many layers is dealing with the overall complexity of interlayer interfaces, configuration details, and management aspects of each layer, as well as the total amount of space, power and equipment spares and repair costs involved to sustain all the layers. There is also an inevitable loss of capacity efficiency from so many layers.

[ Team LiB ]

[ Team LiB ]

2.1 Increasing Network Efficiency

One problem with any stacking of layers, is that the grooming of each layer's traffic into demands on the next lower layer inevitably fails to yield 100% fill. The net utilization multiplies all these fractional fill levels and becomes poorer with increase in the number of layers. For instance, the assignment of logical IP links between routers onto ATM VCs or VPs may typically result in 80% utilization (or less) of the bandwidth actually allocated to those VPs. Similarly the STS-3c fill from the ATM VP cell flows may be only 80%. And the fill of the OC-192s on wavelengths in the optical layer may itself be only another 80%. The net utilization of an OC-192 lightpath with customer payload is the product of each individual trans-layer efficiency, or, in this example: 0.8 x 0.8 x 0.8 which is ~51%. Figure 2-1(a) illustrates this cascading of fractional fill levels down the four-level stack. The overall inefficiency is greater still if we also have ring-based protection at the SONET layer. At its most efficient, SONET ring protection adds another multiplier of 0.5 because working and protection channels are exactly matched.

Figure 2-1. Evolution toward "IP over optics:" four layers down to about "2.5 layers."

Achieving higher overall efficiency is one of the central aims of the mesh-based approach to survivability. Despite what commentators sometimes say, capacity is not free and capacity efficiency is important. Even if capacity was thought to be free (and/or a "glut" of dark fiber abundant in the ground)—planning, installation, testing, and management to bring on additional "lit-fiber" capacity, and integrating it into operational use is certainly not free—so efficiency in its use always pays dividends. An efficient network is inherently more flexible and able to absorb more demand growth or churn before physical additions of new plant and capacity have to be installed. With the Internet driving growth at up to 200% per annum, the problem has often been that capacity simply cannot be laid down fast enough and precisely where it is needed to fully keep up with the growth. In these cases efficiency in the first place, by design, is as good as extra capacity on the ground.

But in fact the simple cost of equipment for added capacity—in the quantities needed is by network operators—is also by no means free.

Since much of this book is aimed at the design of capacity-efficient mesh-survivable networks, it is important to address the often remarked view that "capacity is free."[1]

If so, why would one worry about optimizing capacity efficiency? The basis for this is typically a calculation that assumes a lightpath or fiber bearing, say, a fully-loaded OC-192 and takes the ratio of total cost to total Mb/s carried. The numerical ratio is indeed small. Notwithstanding this, an inter-exchange carrier may have annual budgets for incremental transport equipment of $300 to 600 million. It then depends on the point of view as to whether the cost of capacity, and hence the benefit of capacity efficiency, is significant. If mesh efficiencies can take 20% or more off of the latter kind of annual budget, it should be well-worth the CEOs time and attention.

[1] The point about capacity cost is that it is similar to the notion that "MIPS are free" with computers. Price per MIP may be asymptotically low, but applications always use more. RAM or disk storage is similar. Price per bit is vanishingly small, but the quantity required is always rising. As a result neither is ever insignificant in practice.

But with the four layer stack it is also complex and labor intensive to provision new services. Each layer tends to have its own

management systems. And any one layer can act as a bottleneck to throughput and/or provisioning speed even if all three other layers are

generously provisioned. Thus a simple two layer model for services and transport is attractive: the services layer is IP-based with MPLS and the transport layer is based on a DWDM optical path switched network.

However, in getting to the two layer model, we do not want to abandon the important functions of ATM and SONET in the four layer model.

The role of ATM will be assumed by the IP layer, with MPLS and GMPLS to replace ATM. Similarly, many roles of SONET can be referred down into the optical cross-connecting and transport layer. Certain functions of SONET cannot be eliminated—such as formatting bit streams for physical transmission, framing, error monitoring and so on. They can, however, be realized by "lightweight" implementation of the full SONET standards, or by digital wrapper that puts SONET-like overheads on optical channels (to follow). SONET itself is also being extended in terms of flexibility and payload types by developments such as GFP, virtual concatenation, and LCAS. And there is every expectation that—especially in long-haul networks—SONET ring protection can be replaced by a true mesh-based optical transport network (OTN) using optical cross-connects (OXC) and mesh-survivability schemes. The outcome is expected to be a "two and a half"

layer model conceptualized in Figure 2-1(c), evolving through the intermediate step, (b). This brings the initially idealized view of "services and transport" layers introduced in Chapter 1, much closer to reality. It also makes all of the capacity planning and design models that follow in the book even more directly relevant to industry practice. Let us now look at some of the specific technologies by which this almost-two-layer model of IP services over optical transport can be realized.

[ Team LiB ]

[ Team LiB ]

2.2 DWDM and Optical Networking

In principle, wavelength division multiplexing (WDM) is the same as FDM except that it occurs at optical frequencies—wavelengths in the 1310 and 1550 nm ranges. Several optical carrier signals occupy the same fiber at non-overlapping center wavelengths (or ls). A different payload can be simultaneously modulated onto each optical carrier as long as their laser center frequencies are suitably spaced apart. The main drivers for the deployment of WDM multiplexing are the depletion of fiber capacity in long haul networks due to demand growth and favorable economics of WDM systems relative to alternatives such as installing additional fiber or further upgrading single fiber TDM systems to bit rates beyond 10 Gb/s (OC-192). Although the capacity of early point-to-point WDM systems was limited from 2 to 4 ls, systems with over 1,000 ls have been demonstrated in the laboratory and optical ADMs (OADMs) and optical cross-connects (OXCs) allow wavelengths to be routed and switched in much the same way as timeslots are in SONET networks.

This section is intended to give basic background on WDM concepts, issues, and network elements—to provide a self-contained "setting of the stage" for our subsequent studies. To delve deeper, a range of books on DWDM and optical networking are available. Starting from the most accessible, in the sense of the technical content, [Kart00] is a popular primer on optical transmission and technology. [Gora01]

covers WDM but also SONET, GbE, LAN and WAN networks in a practical high-level way. [Free02] covers optical fiber, lasers, detectors, amplifiers, regenerators, and WDM multiplex devices somewhat more extensively than [Kart00] but at a similar level of technical depth.

[StBa99] and [RaSi98] are more comprehensive textbooks on optical networking including theoretical treatments of topics such as wavelength routing. [Rama01] is a more specialist volume addressing transmission impairment analysis, optical amplifier placement and wavelength conversion technologies. [SiSu00] and [Stav01] are edited compilations of chapters based on specialist research papers on optical network performance issues. Finally, [Dixi03] is a volume that spans both IP and WDM layer considerations with chapters from a variety of experts in both layers. These sources primarily address the challenging basic problem of "working" signal transmission. A few include overviews of protection switching and signaling, or ring concepts for survivability, but generally all are complimentary to the present effort in that they emphasize the technology and basic network planning, leaving the domain of mesh-based survivability strategies to be explored in depth in this book.

2.2.1 Coarse and Dense WDM

In dense wave-division multiplexing (DWDM) numerous laser sources are operated in a closely spaced frequency plan, defining a bank of precisely offset optical carrier frequencies. It is called dense WDM to contrast with the prior technology of coarse WDM. In coarse WDM (CWDM) two to four lasers operate independently at widely separated frequency ranges in the 1550 nm and 1310 nm low-loss windows of the optical fiber. CWDM was intended primarily to achieve pair-gain on a point-to-point basis, effectively doubling or tripling the fiber count between two nodes. CWDM lasers could have a much broader spectral occupancy and looser center frequency stability and relative power levels than DWDM laser sources that must be of extremely narrow linewidth and are usually under active feedback control for absolute center frequency stabilization. In DWDM maintaining relatively equal power levels over all carriers can also be an important and difficult challenge not present in CWDM. Figure 2-2 contrasts coarse and dense WDM wavelength plans and illustrates the concept of DWDM based on tightly spaced and controlled optical carriers in the 1500 nm range where optical fiber attenuation is at its lowest.

Figure 2-2. (a) Coarse and (b) dense wave-division multiplexing.

DWDM systems require precise standardization of the carrier frequencies to use. Current ITU standards define a grid of 81 wavelengths in the "C-band" starting from 1528.77 nm incrementing in multiples of 50 GHz (0.39nm). Commercial systems with 16, 40, 80 and 128 wavelengths per fiber have been announced based on this frequency grid. ITU standards so far only define the 50 or 100 nm spacing grids but laboratory work at 25 nm spacing is occurring and up to 1000 wavelength have been reported on a single fiber [Bell99]. The number and selection of channels implemented on this grid depends on the application requirements such as reach, data rate on the carrier, fiber type, optical filter technologies used, etc., so there is no one set number of wavelength channels associated with all DWDM systems per se.

2.2.2 Optical Amplifiers

The single most important invention that enables DWDM networking is the optical amplifier (OA). An optical amplifier counteracts fiber attenuation over a complete band of lightwave channels at once, without demodulating or in any other way individually processing each optical carrier. In the most common type of OA, an erbium doped fiber amplifier (EDFA), an optical pump laser creates a photon population inversion in a section of erbium doped fiber. The much weaker multi-channel optical input signal from the fiber causes a higher power output of stimulated emission, as in a laser itself, which tracks the input signal thus directly amplifying it in the optical domain. Without this key technology there would be little impetus for DWDM-based optical networking per se because each channel would have to be

individually detected and regenerated or amplified each time fiber loss accumulated above threshold. There would still be multiplexing gain in the use of the fiber itself, but no corresponding relief in the amount of physical equipment needed at each regenerator or amplifier station along the fiber route. The EDFA has been so successful that the term is often used as a synonym for OA itself. But there are other OA technologies, such as the semiconductor optical amplifier (SOA) and erbium doped waveguide amplifier (EDWA).

2.2.3 Regenerators

The optical amplifier is an analog device. It functions as a broadband linear amplifier of incoming light power to offset fiber attenuation or other insertion losses in the optical path. In practice, OAs are neither perfectly linear nor noise-free. They add noise as well as boost any

incoming noise with the signal. Other fiber and amplifier related impairments accumulate in the optical path such as chromatic dispersion, polarization mode dispersion, and types of crosstalk. The important point here is that these impairments add up to a point where

regeneration (as opposed to amplification) is required. In regeneration, more specifically in a "3R" regenerator, each channel is individually demodulated from its optical carrier and converted to an electrical form where its bit-timing is extracted and used to sample the binary state of each symbol. Jitter is removed from the recovered clock, new 1/0 symbols generated, and these are remodulated onto a new outgoing carrier wavelength. "3R" thus refers to retiming, regenerating and retransmitting.

Regeneration is a per-channel process involving electro-optical conversion and high speed electrical processing of each channel individually and is therefore more costly than optical amplification. Often the regeneration function needed along an optical path would be provided by optical-electrical-optical (o-e-o) cross-connects along the route of the path. The need for regeneration is one of the reasons that "transparency" (optical-only processing without wavelength conversion) is likely to be limited in optical networks. As long as there is a need for regeneration, it makes sense for at least some of the cross-connects in the network to be electrical-core devices as opposed to purely passive optical (o-o-o) space switches. Any time the signal has been demodulated to the electronic domain for regeneration or switching, we can also change wavelengths as needed because the signal is applied to a new transmitting laser to return it to the optical domain. Less often a bank of stand-alone regenerators may be needed on long-haul facility routes, contributing considerable extra cost to the signal path. With present OA spacings of around 60 to 80 km, regenerator spacings are typically around 550 to 600 km. Note that in the design of the DWDM link these spacings are not independent. If OAs are more frequent, optical signal to noise ratio will be better preserved allowing for a longer distance between regenerators (and the converse).

For a variety of network planning purposes, the detailed design of a transmission span between nodes is usually summarized into a single incremental cost for adding a channel to the span as a whole or a set of costs associated with modular capacity addition options along the entire span. Such cost coefficients summarize the net effect of the detailed transmission design efforts, taking into account each span's different length and exact equipment types and spacings and each of the technology options available to the planner. The per-channel cost may or may not include a pro-rated allocation of all the "get started" costs associated with the acquisition of the route, installing ducts, cables and equipment huts, etc., before the first channel is turned up.

2.2.4 Optical Add/drop Multiplexers (OADMs)

The structure and function of an OADM almost completely follows from its SONET ADM predecessors, although a variety of different technical implementations exist for selecting the optical channels to add or drop. In principle an OADM can be based on electronic detection of payloads, electronic add/drop, and remodulation of pass-through and add signals onto new outgoing wavelengths. Such an OADM is capable of full wavelength conversion as well as add/drop and pass-through functions. But especially in an OADM it is desirable to leave pass-through channels in the optical domain. Many OADM designs therefore tend to be based on passive, purely-optical add/drop/through channel filters without any wavelength conversion. Only add/drop channels need to go through e/o and o/e conversion respectively.

In addition, because of the technical challenges of selecting or inserting single wavelengths, OADMs may be based on a waveband add/drop principle, where groups of 4 or 16 wavelengths are handled as a whole for add or drop purposes and all other wavelengths pass through the OADM to the other fiber. Also, because of the technological challenges involved, an OADM may or may not be reconfigurable

In addition, because of the technical challenges of selecting or inserting single wavelengths, OADMs may be based on a waveband add/drop principle, where groups of 4 or 16 wavelengths are handled as a whole for add or drop purposes and all other wavelengths pass through the OADM to the other fiber. Also, because of the technological challenges involved, an OADM may or may not be reconfigurable