Terabit networks support transmission rates of at least one trillion(1012) bits/second (Tb/s)1. These networks are becoming more common because they provide the capacity and bandwidth needed to meet increasing customer demand for data and voice communications, and to support future Internet growth of high quality video and e-commerce applications. When properly designed they can also reduce latency for Long Haul Network (LHN) traffic, reduce the time needed for new circuit provisioning, and reduce overall network management complexity as well.
This overview of terabit networks consists of the following subsections:
- Market and Service Drivers
- Challenges and Requirements
- Service Network Architecture - Overview
- Service Network Architecture - Detailed View
- Core Optical Network (CON) Traffic Provisioning & Management
- Terabit Optical Technologies
- For Further Reading
Market and Service Drivers
Demand for terabit networks is being driven by a variety of factors. Residential users want more bandwidth for video, electronic gaming and music applications. Even though today’s networks can often provide up to 10 Mb/s bandwidth per household, tomorrow’s HDTV and multi-channel video applications will require ten times that rate.
Moreover, Hollywood studios are now predicting that the "future of cinema" will require real-time streaming of Super High-Definition (SHD) video. For picture quality equivalent to film, the "SHD-4K" technology will require at least four times the bandwidth needed for HDTV - meaning at least 1 Gb/s per household! Once you can imagine a neighborhood of a thousand households, each demanding 1 Gb/s of bandwidth, it’s easy to see how terabit networks could become commonplace. Also, extrapolating this to an entire city gives an inkling of the transmission rates that will be needed for the next generation of IP networks.
Small businesses will also require increased network access speeds to support the proliferation of interactive services for customers placing orders via IP-Phones, Video-Phones, etc. Similarly, telecommuters working from home will want VPN access to enterprise networks. As one of many examples, this would permit running complex equipment performance simulations on enterprise mainframe computers. Both of these terabit network users will likely demand Service Level Agreements (SLAs) that offer shorter out-of-service times, lower transfer delays, more well-defined QoS mechanisms, and enhanced traffic admission controls when compared with today’s networks.
Large Corporate Enterprises have all of the above requirements and in addition they will require:
- Higher speeds
- Support for more complex point-to-point or point-to-multipoint topologies
- Ability to handle mixed-traffic and protocols
- Support for generalized Virtual Private Network (VPN) configurations
- Resilient back-up support and automated contingency configurations to protect against failures.
Challenges and Requirements
The above market and service drivers give rise to some unique terabit network challenges and requirements. Chief among these as described in more detail below are: network scalability, flexibility, efficiency and transparency, improved network management & operations costs, multi-protocol support, rapid service recovery, and authentication, authorization and accounting.
Terabit network applications are characterized by unpredictable client traffic demands combined with stringent requirements on Quality-of-Service (QoS). Traditionally, traffic planners could consider capacity growth in three-, five- and ten-year increments. Today, the rapid and explosive growth in web video, mobile messaging, wi-fi and wi-max applications, means time frames as short as six months must also be considered. Thus, graceful scalability is a prime terabit network requirement.
- Flexibility, Efficiency and Transparency
From a customer service perspective, terabit network platforms must be very flexible, enabling clients to increase service velocity on demand at any time, and from any location. The networks must also efficiently accommodate a diverse set of both differentiated service offerings (e.g., based on priority, resiliency, etc.), and wide-ranging traffic characteristics (e.g. real-time traffic, legacy protocols, high peak traffic, etc.).
- Improved Network Management & Operations Costs
Today’s users not only want more bandwidth for their money, they demand simpler and low-cost network management & operations procedures. Hence terabit network equipment suppliers must offer both operational savings (lower power consumption, reduced management complexity, smaller footprint), and support modular deployments, and continuous growth.
As new services proliferate, terabit network operators are looking to new "de-layered" and transparent network infrastructures to support all customer services across all customer locations, while providing reduced transmission and operations overhead for a variety of protocols.
SONET/SDH network providers using Resilient Packet Ring (RPR) technology built to meet the IEEE 802.17 RPR standard are accustomed to a maximum dual-ring-topology restoration time of 50 mSec. Some network equipment venders offer even faster recovery times. New terabit network technologies must offer at least this level of protection or better.
- Authentication, Authorization and Accounting
Authentication, authorization and accounting are the known as the "triple-A" of network security. A key terabit network infrastructure issue is how to provide the servers and security mechanisms to ensure that no single person or resource can gain network access without proper authorization.
Service Network Architecture - Overview
Figure 1 shows a layered architecture model for terabit networks that is emerging for enterprise and public service provider infrastructures alike. The lowest layer supports multi-service access for all types of data, voice, and video over a single packet-cell-based infrastructure. The benefits of multi-service access are reduced OPerating EXpenses (OPEX)2, higher performance, greater flexibility, integration and control, and faster service deployment.
The heart of the architecture is a Core Optical Network (CON) which serves to interconnect the multi-service access points with the service platform. Since per-bit profit margins will still be constrained by aggressive competition, the CON must be designed with minimal complexity to reduce costs, while still flexibly and efficiently supporting multi-service transport.
Figure 1. Layered Terabit Network Service Architecture - Overview
CON packet forwarding overhead is greatly reduced through use of Multi-Protocol Label Switching (MPLS) technology. Internet Protocol (IP) packets have a field in their header containing the address to which the packet is to be routed. Traditional routing networks process this information at every router in a packet's path through the network. Using MPLS, however, when the data packet enters the first router, the header analysis is done just once and a new label is attached to the packet. Subsequent CON MPLS routers can then forward the packet by inspecting only the new label.
In MPLS terminology, the CON routers are classified into two categories: high-performance packet classifiers called Edge Routers or Label Edge Routers (LERs) that apply (and remove) the requisite MPLS labels, and core routers that perform routing based only on Label Switching and are also called Label Switch Routers (LSRs).
MPLS technology supports both traffic prioritization and QoS, and it can be used to carry many different kinds of traffic, including IP packets, ATM, SONET, and Ethernet. IP will likely be the near-universal technology used to implement the service layer, and Dense Wavelength Division Multiplexing (DWDM) will be used to increase bandwidth over existing fiber-optic backbones.
Finally, the CON will link to the service platform which will in turn support execution of a variety of distributed applications, network management processes and signaling and control functions, as well as access to a diversity of information content types.
Service Network Architecture - Detailed View
Figure 2. Layered Terabit Network Service Architecture - Detailed View
Figure 2 shows the layered architecture model for terabit networks in more detail. It consists of the following parts:
Personal Area Networks (PANs)
Areas one to three meters in extent which are serviced by wireless technologies such as Bluetooth, Zigbee and Wireless Universal Serial Bus (WUSB).
Local Area Networks (LANs)
Link user premises to the first network node. Next generation LANs will be optical and support 100 Gb/s; one terabit Ethernet is being planned for 2010-12.
Metropolitan Area Networks (MANs)
Provide corporate connections inside the city. Here fiber optics and Ethernet protocol is the favorite as a MAC layer, although SONET/SDH is also in use. Terabit network technology will initially have the most impact on MANs and Long Haul Networks (LHNs).
Distribution and Transport Network (LHN)
Is the inter-city equivalent to the "express train" that transports many people through long distances. But if you don’t live in a city you’ll need to access a "local light rail" line somewhere close to home. That’s the distribution network that could be "along the way" of the express route, or a complementary route "orthogonal" to the express line. Ideally, both networks will be planned together and use the same technology.
Regional Area Networks (RANs)
Are useful for localized services from a regional carrier, a local enterprise, or a county or group of cities. RANs are needed for services that exceed geographic boundaries such as those for international corporations, national services, federal police networks, etc.
The main topological design problem in terabit networks is deciding where to locate multi-service access nodes, and how to provision and manage traffic flexibly and efficiently as described in more detail in the next section.
Core Optical Network (CON) Traffic Provisioning & Management
Provisioning and management of terabit network traffic must be done simply and efficiently to maximize network throughput, reduce buffer size and processing power, and to minimize delay due to memory allocation and packet processing at CON nodes. Multi-service access nodes and MAN transport will depend on Ethernet Layer 2 (L2) aggregation techniques whereby frame labels such as Virtual LAN (VLAN) tags or MPLS Permanent Virtual Circuits (PVCs) support a finer level of granularity than provided by the Long Haul Network (LHN).
VLAN tags and PVCs connect customer IP routers to an IP service switch at the CON’s edge. Residences, small businesses and small-to-medium enterprises with links to multi-service access nodes will migrate to Passive Optical Networks (PONs) to-the-curb (or to-the-Building), and will be terminated using a variety of "last mile" technologies including copper, wireless3 and fiber.
Link Capacity Adjustment Scheme (LCAS) with Virtual Concatenation (VC)
The Optical Internetworking Forum (OIF) defines the Optical User-Network Interface (UNI) that provides an interface by which a client may request services from an optical network. The SONET/SDH Link Capacity Adjustment Scheme (LCAS) includes automated traffic provisioning by means of Virtual Concatenation (VC) in a variety of sizes and can automatically adjust the transmission capacity seen by the end user. Automated connection provisioning opens the way to offer additional services such as intelligent protection and restoration of back-up links without requiring expensive hardware components to achieve redundancy.
Traffic Grooming
Multiplexing frames at network ingress points compromises efficiency when the network has many entry points. Accommodating frames inside faster and longer frames requires a tradeoff between load flexibility and efficient use of link capacity. Newer optical grooming technologies support traffic flows that minimize the number of add/drop operations. Admission control enables client traffic to be controlled based on a mutually agreed-upon Service Level Agreement (SLA). Traffic management depends on queuing and scheduling procedures for the incoming traffic flows that were authorized by admission control. LCAS/VC offers network providers flexibility inside virtual circuits to accommodate client traffic fluctuations and add/drop of circuits without changing the network physical structure.
Distributed and Automated Network Management
Terabit networks require a large number of measurements and traffic data that must be processed by the NMSs to prevent traffic overloads. The huge volume of this data can result in long delays before the network traffic is brought under control. Moreover, a central node or link failure can readily erode the QoS on a large portion of any network. Traditional NMSs have centralized control. However, the terabit network’s increased complexity, equipment diversity, need for flexible service provisioning, topology reconfiguration and protocol updates, as well as traffic fluctuations, mandate a distributed and automated approach to network management.
Current SONET/SDH networks use manual processes and Network Management Systems (NMSs) to implement optical connections from one location to another. Turn-around time to provision a new connection can take as long as six weeks, and the configuration process can take several hours, especially if more than one carrier is involved. While this may be acceptable for LHN where the end nodes are cities and change infrequently, it is by no means responsive enough for MAN solutions where end nodes are enterprise branches or connections between enterprises.
Optical links to support MANs require a dynamic automated provisioning system that offers short turnaround times, flexible scalability, fine traffic granularities, and is amenable to frequent changes. Recently, dynamic provisioning protocols have emerged that let carriers establish connections not only within a single carrier’s territory, but can also provide dynamic provisioning across multiple carriers on an end-to-end basis.
Terabit Optical Technologies
Wavelength Division Multiplexing (WDM) has dominated fiber-optic transmission technology since the development of tunable lasers. Two WDM technologies were developed: Dense Wavelength Division Multiplexing (DWDM) for long haul transmission and Coarse Wavelength Division Multiplexing (CWDM) for metropolitan transmission. The first is very precise and very costly but supports hundreds of optical channels; the second is inexpensive and can be implemented on a variety of physical media but supports only 18 optical channels. CWDM is the appropriate technology for PON local access networks and DWDM is the right technology for Distribution and Transport Network inside the LHON (Long Haul Optical Network). CWDM can be easily implemented with point-to-point or point-to-multipoint topologies, but DWDM requires that optical channels be provisioned on specialized nodes.
Synchronous Optical Network (SONET) and the Synchronous Digital Hierarchy (SDH) offer similar packet data containers of 155 Mb/s, 622 Mb/s, 2.25 Gb/s, 10 Gb/s, and 40 Gb/s. The next logical step would result in the evolution of these protocols to terabit rates as multiples of 1.3 Tb/s. At least 100, 1.3 Tb/s channels, can be placed inside a fiber-optic cable consisting of 20 fibers. If ten fibers are used to support one direction of transmission, and ten fibers the opposite direction, the resulting fiber cable capacity is equal to (10 fibers * 100 channels * 1 Tb/s per channel), or 1000 Tb/s (1 Petabit per second).
In December 2006, the Ethernet Alliance (www.ethernetalliance.org) delegated the IEEE 802.3 Standards Project to the High Speed Study Group (HSSG). This group is forecasting that 100 Gb/s Ethernet could be the new IEEE standard for 2010. As bandwidth demands continue to require faster access networks, and hardware manufactures implement ever faster chip-sets, the next logical step would be a 1 Tb/s Ethernet protocol.
1 Note: A terabit is about 10% less than a "tebibit" (sometimes abbreviated as a "tibit"), which is equal to 2
40 or 1,099,511,627,776 bits.
2 Operating Expenses (OPEX) are the amount paid for asset maintenance or the cost of doing business, excluding depreciation. Earnings are distributed after operating expenses are deducted.
3 Wireless technologies are not discussed here because it’s very difficult for wireless to break the 100 Mb/s barrier. In the future, better compression and modulation techniques may increase wireless speeds to 1 Gb/s, but that will still be too slow for many future services. Of course, wireless will continue to be widely used to support mobility services.
