MBONE, the Multicast BackbONE

Mike Macedonia and Don Brutzman

Naval Postgraduate School

Introduction.

The joy of science is in the discovery. It was a year ago when we heard that the JASON Project, an underwater exploration educational program supported by Woods Hole Oceanographic Institution, was showing live video over the Internet from an underwater robot in waters off Baja, Mexico. Our group here at the Naval Postgraduate School (NPS) furiously tried to figure out how to receive that video signal. We worked for several days to gather the right equipment, contact the appropriate network managers, and get hardware permissions from local bureaucrats, only to find that an antenna uplink on the JASON support ship had flooded a few hours before we became operational. Despite this disappointment we were not discouraged because we had discovered how to use the Internet's most unique network, MBONE.

MBONE stands for Multicast Backbone, a virtual network that has been in existence for about three years. The network originated from an effort to multicast audio and video from the Internet Engineering Task Force (IETF) meetings. MBONE today is used by several hundred researchers for developing protocols and applications for group communication. Multicast is used because it provides one-to-many and many-to-many network delivery services for applications such as videoconferencing and audio that need to communicate with several other hosts simultaneously.

Multicast networks.

Multicasting has existed for several years on local area networks such at Ethernet and FDDI. However, with Internet Protocol (IP) multicast addressing at the network layer the service group communication can be established across the Internet. IP multicast addressing is an Internet standard developed by Steve Deering (Request For Comment RFC-1112) and is supported by a number of workstation vendors, including Sun and Silicon Graphics Inc. Categorized officially as an IP Class D address, an IP multicast address is mapped to the underlying hardware multicast services of a local area network.

The reason that MBONE is a virtual network is that it shares the same physical media as the Internet, though it must use a parallel system of routers that can support multicast (e.g. dedicated workstations running with modified kernels and multiple interfaces) augmented with "tunnels". Tunneling is a scheme to forward multicast packets among the islands of MBONE subnets through Internet IP routers which typically do not support IP multicast. This is done by encapsulating the multicast packets inside regular IP packets.

Bandwidth.

The key to understanding the constraints of MBONE is thinking about bandwidth. The reason why a multicast stream is bandwidth-efficient is that one packet can touch all workstations on a network. Thus a 125 Kbps video stream (1 frame/second) uses the same bandwidth whether it is received by one workstation or twenty. That is good. However that same multicast packet is prevented from crossing network boundaries such as routers or bridges. The reasons for this restriction are religious and obvious: if a multicast stream which can touch every workstation could jump from local network to local network, then the entire Internet would quickly become saturated by such streams. That is very bad! Thus the MBONE scheme encapsulates multicast streams into unicast packets which can be passed as regular Internet protocol packets along a virtual network of dedicated multicast routers (mrouters) until they reach the various destination local area networks. The use of dedicated mrouters segregates MBONE packet delivery, protecting standard network communications such as mail and telnet from MBONE experiments and failures. Once properly established, an mrouter needs little or no attention. Given this robust distribution scheme, responsible daily use of the MBONE network consists only of making sure you don't overload your local or regional bandwidth capacity.

Networking details.

When a host on an MBONE-equipped subnet establishes or joins a group it announces that event via the Internet Group Management Protocol (IGMP). The multicast router on the subnet forwards it the other routers in the network. MBONE sessions use a tool developed by Van Jacobson of Lawrence Berkeley Laboratories called sd (session directory) to display the announcements by multicast groups. sd is also used for launching multicast applications and for automatically selecting an unused address for any new groups.

Groups are disestablished when everyone leaves, freeingup the IP multicast address for reuse. The routers occasionally poll hosts on the subnets to determine if any are still group members. If there is no reply by a host, the router stops advertising that hosts group membership to the other multicast routers.

The routing protocols for MBONE are still immature andtheir ongoing design is a central part of this network experiment. Most MBONE routers employ the Distance Vector Multicast Routing Protocol (DVMRP) which is commonly considered inadequate for rapidly changing network topologies because routing information propagates too slowly. A multicast extension to the Open Shortest Path (MOSPF) link-state protocol has been proposed by John Moy of Proteon Inc. that addresses this problem. However, with both protocols each router must compute a source tree for each participant in a multicast group. MBONE is currently small enough that this restriction is not a problem. However, for a large network with constantly changing group memberships such routing techniques are expected to be computationally inefficient.

Topology and Scheduling.

The MBONE topology and the scheduling of multicast sessions must be actively managed by the MBONE community to minimize congestion. Approximately 400 sites worldwide are currently MBONE members. MBONE protocol developers are currently experimenting with automatically pruning and grafting subtrees, but for the most part uses truncated broadcasts to the leaf routers. The truncation is based on the setting for the time-to-live (ttl) field in a packet which is decremented each time the packet passes though a router. A ttl value of 16 would limit multicast to a campus, as opposed to a value of 100 which might send it to every subnet on the entire MBONE (about thirteen countries).

These issues can have a major impact on network performance. For example, a default video stream consumes about 128 Kbps (kilobits per second) of bandwidth, which is almost 10 percent of a T1 line (a common site-to-site link on the Internet). Several simultaneous high-bandwidth sessions might easily saturate network links and routers. This problem is compounded by the fact that general purpose workstation routers typically used by the MBONE are normally not as fast or as robust as the dedicated hardware routers used in most of the Internet.

Protocols.

The magic of MBONE is that teleconferencing can be done in the hostile world of the Internet where variable packet delivery delays and limited bandwidth play havoc with applications that require some real-time guarantees. It is worth noting that only a few years ago puting audio and video across the Internet was considered impossible. Development of effective multicast protocols disproved that widespread opinion. In this respect MBONE is like the proverbial talking dog: it's not so much what the dog has to say, it's more that the dog can talk at all that is amazing.

In addition to the multicast protocols, MBONE applications are using the Real Time Protocol (RTP) on top of User Datagram Protocol (UDP) and IP. RTP is being developed by the Audio-Video Transport Working Group within the IETF. RTP provides timing and sequencing services; permitting the application to adapt and smooth out network-induced latencies and errors. The end result is that even with a time-critical application like an audio tool, participants normally perceive conversations as if they are in real-time, even though there is actually a small buffering delay to synchronize and sequence the arriving voice packets. Protocol development continues. Although operation is usually robust, many aspects of MBONE are still considered experimental.

Data Compression.

Another aspect of this research is the need to compress a variety of media and to provide privacy through encryption. Several techniques to reduce bandwidth include Joint Photographic Experts Group (JPEG) compression and the ISO standard H.261 for video. Visually this translates to velocity compression: rapidly changing screen blocks are updated much more frequently than slowly changing blocks. Encodings for audio include Pulse Coded Modulation (PCM) and GSM. Ouside of the concerns for real-time delivery, audio is a difficult media for the MBONE and teleconferencing in general because of the need to balance signal levels for all the parties who may have different audio processing hardware (e.g. microphones and amplifiers). Audio also generates lots of relatively small packets, which are the bane of network routers.

Application Tools.

Besides basic networking technology, MBONE researchers are developing new applications that typify many of the goals associated with an "information superhighway." Video, audio, and a shared drawing whiteboard are the principal applications, provided by software packages called nv (net video), vat (visual audio tool) and wb (whiteboard). The principal authors of these tools are Ron Frederick of Xerox Palo Alto Research Center (PARC) for nv and Van Jacobson of Lawrence Berkeley Laboratory for vat and wb. Each of these programs is available in compilable or executable form without charge from various anonymous ftp sites on the Internet. Working versions are available for Sn, SGI and VMS architectures with ports in progress for HP-UX and Macintosh. No DOS, OS-2 or Windows versions are currently available although ported tools can be found for 386 boxes running the (free) 386bsd Unix. Pointers to all public application tools are included in the FAQ.

Additional tools are also available or under development. Winston Dang of the University of Hawaii has created imm (Image Multicaster Client), a low-bandwidth image server. It is typically used to provide live images of planet Earth from geostationary satellites at half-hour intervals in either visible or infrared (IR) spectra. Article author Mike Macedonia has ported the IEEE Distributed Interactive Simulation (DIS) protocol to enable live interaction between virtual worlds as an MBONE communications tool. Other researchers are experimenting with using graphics workstation windows as image drivers. Future network news distributions may be multicast to reduce overall network loading and speed news delivery.

Events.

Many of the most exciting events on the Internet appear first on MBONE. Perhaps the most popular is NASA Select, the NASA in-house cable channel broadcast during space shuttle missions. Be warned that this might be a work stopper! It is hard to describe the excitement of seeing one astronaut hold another astronaut by the boots to repair a satellite, all live from 150 miles above the earth. Conferences on supercomputing, Internet Engineering Task Force, scientific visualization and many other topics have appeared, often accompanied by directions on how to download PostScript copies of presented papers and slides from anonymous ftp sites. Radio Free VAT is a community radio station whose DJ's sign up for air time via an automated server (vat-radio-request@elxr.jpl.nasa.gov). Xerox PARC occasionally broadcasts lectures by distinguished speakers. Internet Talk Radio (Carl Malamud, info@radio.com) has presented talks by Larry King and "Geek of the Week." Remote learning has brought expertise over long distances and multiplied training benefits. Default MBONE audio and video channels are available for new users, experimentation and advice from more experienced users.

Groupwork on groupware.

The MBONE community is active and open. Work on tools, protocols, standards, applications and events is very much a cooperative and international effort. Feedback and suggestions are often relayed to the entire MBONE mailing list (as an example, this article was proofed by that group). Cooperation is essential due to the limited bandwidth of many networks, in particular transoceanic links. So far no hierarchical scheme has been necessary for resolving potentially contentious issues such as topology changes or event scheduling. Distributed problem solving and decision making has worked on a human level just as successfully as on the network protocol level. Hopefully this decentralized approach will continue to be successful even in the face of rapid addition of new users.

Cost of admission.

The first thing you need to get on MBONE is the willingness to study and learn how to use these new and fast-moving tools. The second thing you need is bandwidth. Here at NPS we run MBONE tools on workstations connected via Ethernet (10 Mbps). Off-campus links are via T1 lines (1.5 Mbps). We have found that bandwidth capacities lower than T1 will result in network crashes and thus appear unsuitable for MBONE.

Given adequate network bandwidth, you now need a designated MBONE network administrator. It typically takes one to three weeks for a network-knowledgeable person working part-time to establish MBONE at a new site. Setup is not for the faint of heart, but all of the tools are documented and help is available from the MBONE list. Read the Frequently Asked Questions (FAQ) a few times and ensure that software tools and multicast- compatible kernels are available for your target workstations. Subscribe to the mail list in advance so that you will be able to ask questions and receive answers. Figure 1 shows the various worldwide MBONE list subscription request addresses. After subscribing, read the FAQ again.

To receive multicast packets on your local area network, you will need to configure an mrouter which strips off packet encapsulation. This can be a dedicated router. A more popular approach is to take an old slow cast-off workstation and equip it with two Ethernet cards. Two network cards are needed, one to receive the upstream tunnel, and the other to distribute downstream multicast packets. Obtain and load the application software tools. You are now ready to put multicast on your local area network.

Note that these tools can also work in isolation between workstations on a single local area network without any mrouter. We recommend that you test the application tools locally in advance (before going through the mrouter effort) to see if they are compatible and match your expectations.

Once you are connected, pass along any lessons learned to the tool authors or the MBONE list. Later show your overall network site administrator something spectacular on MBONE (such as a live spacewalk) and then make sure that your site is programming funds to increase your network bandwidth. Demands on network bandwidth are significant and getting more critical. You might consider Tengdin's First (and Only) Law of Telecommunications: "The jump from zero to whatever baud rate is the most important jump you can make. After that everyone always wnts to go straight to the speed of light."

Caveats.

Some problems still exist and a lot of work is still in progress. The audio interface takes coaching and practice. Leaving your microphone on by mistake may override everyone else since only one person can talk at a time. You will need a video capture board in your workstation to transmit video, but no special hardware isneeded to receive video. One frame per second video seems pretty slow (standard video is 30 frames per second), but in practice it is surprisingly effective when combined with phone-quality voice. One user blasting a high-bandwidth video signal (greater than 125 Kbps) can cause severe and widespread network problems. Controls on access to tools are rudimentary and security is minimal; for example, a local user might figure out how to listen through your workstation mike (unless you unplug it). Audio broadcast preparations are often just as involved as video broadcast preparations. Network monitoring tools are not yet convenient to use. Internet bandwidth is still inadequate for MBONE in many countries. On one occasion a local topology change at our school caused a feedback loop that overrode the NASA Select audio track. Although plenty of people were willing to point out the symptoms of our error (!) it was not possible for the rest of the network to cut off the offending workstation cleanly. More situations will undoubtedly occur as the MBONE developers and users learn more and continue to improve the tools. Expect to spend some time if you want to be an MBONE user.

The future.

It is not every day that someone says to you "Here is a multimedia television station that you can use to broadcast from your desktop to the world." These are powerful concepts and powerful tools that tremendously extend our ability to communicate and collaborate. These tools are already changing the way people work and interact on the net. See you later!