X. AUXILIARY DATA TRANSPORT, 22 March 1996
X.1 OVERVIEW
The Auxiliary Data Transporter is the portion of CU-SeeMe that provides a mechanism for the exchange of data that is neither video nor audio; since those data streams are the "primary" constituents of a video conference, everything else can be considered "secondary" or "auxiliary". While there are some situations within a video conference where two participants need to exchange data, Aux Data was more particularly designed to send data from a given participant to all others. Because CU-SeeMe uses UDP for sending and receiving packets, the Aux Data Transporter has the added problem of providing reliability and doing so with an unknown number of conference participants. These requirements are different than those for video and audio and have necessitated the development of several approaches to error detection and recovery.
As an example, during a normal conference the speaker may use an overhead projector to display an image that helps illustrate the speech. At other times, the speaker may provide hand-outs, either during the conference or before it starts. These items are aids to the speech; they are "auxiliary" in that they are not crucial to the flow of information but instead merely help that flow.
In a two-party conference, there might also be a need to exchange additional information. In a conference between two doctors, for example, it might be very important that they both be able to examine some data such as an X-ray image or a photograph of a tissue sample. In these cases, the data might be crucial even though it is "auxiliary" to the conference itself.
In general, the Aux Data Transporter might be described as using "broadcast with negative acknowledgement and availability reports". The sender of some Aux Data does not become concerned about its arrival (or lack thereof) until a recipient indicates that the data did not entirely arrive; at that point the sender will re-send the missing portions of the data. For cases in which the Aux Data is in fact crucial, the Transporter also provides for a reliable transmission. Note that the "broadcast" aspects of Aux Data are actually handled by the Reflector program, just as they are for CU-SeeMe's video and audio datastreams.
A few definitions would seem to be in order:
An "Aux Data Application" is a program that uses Aux data; in general, such an application will both send and receive data, but that is not strictly necessary. An application may be built into CU-SeeMe or it may attach via a "plug-in" interface. The application can call a Transporter routine to identify itself as being ready to process some "Type" of Aux Data.
An "Aux Data Item" is a collection of contiguous bytes sent and received as a logical entity. At present, an entire item must fit in to its sender's memory but it does not necessarily have to fit within its receivers' memory. There is no other practical limit to the size of an item; in particular, the item's size is not related to any network data packet size. Each item has a 4-byte type value that, in effect, identifies the receiving Aux Data Application that will process the item.
Each item gets broken up into "Aux Data Segments" for transmission over the network. The segments are prefixed with information about the segment and the item such that the segments can be re-assembled by the receivers to re-construct the item. As the segments traverse the network, some may get lost and will therefore have to be sent again. Segments may arrive out of order, although at present they are sent in ascending byte order.
In order to control the flow of Aux Data, the Aux Data Transporter also sends and receives "Aux Data Supervisory Packets", which are distinct from the "Aux Data Data Packets" that contain the items' segments. These supervisory packets are not visible to any Aux Data Application.
X.2.1 PACKET FORMAT -- Data packets At present, Aux Data is sent in its own packets, but it might someday be packed into unused portions of audio or video packets. To make that change simpler, each Aux Data data packet has its own special header; at present, this header follows the standard "CuSeeMePacketHeader".
The Aux Data header ("struct AuxDataHeader") and the data (if any):
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | Version | HdrLen | Opcode | Priority |
+--------------+--------------+--------------+--------------+
04h | Prune |
+--------------+--------------+--------------+--------------+
08h | Status | UNUSED | SegmentLength |
+--------------+--------------+--------------+--------------+
0Ch | Type |
+--------------+--------------+--------------+--------------+
10h | ItemNumber |
+--------------+--------------+--------------+--------------+
14H | ItemID |
+--------------+--------------+--------------+--------------+
18H | ItemLength |
+--------------+--------------+--------------+--------------+
1Ch | SegmentOffset |
+--------------+--------------+--------------+--------------+
20h | (data for this segment, if any) |
. ... .
+--------------+--------------+--------------+--------------+
(Note that the header is shown as it appears on the network, reading from left to right and from top to bottom; this is in "Big Endian" format as used on 680x0 Macintoshes. Also note that the labels shown above are not exactly as they appear in the code; the real ones are longer and more "self descriptive" but would not fit well into the diagram.)
The various fields are used as follows:
00h - Version (1 byte - unsigned char)
This field provides a means for supporting various versions of the Aux Data packets or, at least, for preventing obsolete versions from causing crashes. If it proves necessary or desirable to change the format of the Aux Data packets, then this field will have a new value; it is presently 0x00.
01h - HdrLen (1 byte - unsigned char)
This field contains the length of the AuxDataHeader itself. If we later need to add a new field to the header, and if we can safely add it at the end of the header, and if we can safely process older packets that do not have that new field, then we can simple add it and this "HdrLen" value will automagically still point to the data area within the packet.
02h - Opcode (1 byte - unsigned char)
This field is a copy of the auxOpcode parameter given by the application when it called "SendAuxDataItem". At present, it can have these values:
ADS_OP_SEND (0) Normal "best try" transmission. ADS_OP_SEND_WITH_ACK (1) Require positive acknowledgment. ADS_OP_SEND_ONCE (2) Send just once, with no retries.03h - Priority (1 byte - unsigned char)
This field is a copy of the auxPriority parameter given in the call to "SendAuxDataItem". At present, it can have the following values (at this writing, and in the spirit of egalitarianism, the field is ignored):
ADS_PRIORITY_LOW (-128) Not an important item. ADS_PRIORITY_NORMAL (0) A normal item. ADS_PRIORITY_HIGH (+127) Very important item.04h - Prune (1 byte - unsigned long)
This field is a 32-bit value containing a single bit that corresponds to this item's actual "type". The bit can be used by a reflector to compare with a master bit-map for each recipient; the reflector can then "prune" back unwanted aux data streams. In some implementations, not all of the prune bits might be in use; in that case, the "interesting" ones will be in the right-hand side of the field; a "Big Endian" value of 0x00000001 represents the first prune bit. Note that the prune bits are set by the sender but are not checked at all by the Aux Data Transporter; they are there simply to make life easier for the reflector program.
08h - Status (1 byte - unsigned char)
This field is a general-purpose status field. At present, it can have these values:
ADR_STATUS_DATA (0) A normal data segment. ADR_STATUS_RETRY (1) A retry data segment. ADR_STATUS_INCOMPLETE (2) The item has been cancelled.0Ah - SegmentLength (2 bytes - unsigned short)
This 16-bit field contains the actual length of the data portion of the packet.
0Ch - Type (4 bytes - unsigned char array, but processed as an unsigned long)
This 4-byte field is a copy of the auxType field from the original call to "SendAuxDataItem" and is the sending application's identifier for the intended recipient application. The type value can be freely chosen by the application but must be unique among all Aux Data applications that might be used within a given conference. Cornell University has reserved all values beginning with the ASCII characters "CU".
10h - ItemNumber (4 bytes - signed long)
This field is an internal sequence number that starts at 1 and increases monotonically for all normal send requests. For "send with acknowledgment" requests, it starts at -1 and decreases monotonically. There is probably no need for an application to be concerned about this number, which is used solely for error recovery by the Aux Data Transporter.
14h - ItemID (4 bytes - unsigned long)
This 32-bit field is a copy of the auxItemID field from the original call to "SendAuxDataItem" and is the sending application's identifier for this specific item. In general, applications will probably let this value increase by 1 for each item, but that is strictly up to the application; it is not checked by the Aux Data routines.
18h - ItemLength (4 bytes - unsigned long)
This 32-bit field is a copy of the auxItemLength field from the original call to "SendAuxDataItem" and is the total length of all of the data for this item.
1Ch - SegmentOffset (4 bytes - unsigned long)
This 32-bit field is the 0-origin offset of this segment within the item. Typically, the recipient's data-handler will use this as an offset within the item's buffer when it moves the segment's data into that buffer. Because of network delays and errors, segments may not arrive contiguously or in order, and so the data-handler will have to be able to accept the segments individually.
20h - Data (0 or more bytes)
This area is the actual data comprising this segment of this item. While it is true that the segments are sent in ascending order within the item, they may arrive out of order and may overlap segments already received. It is up to the receiving application to re-assemble the item out of the various segments.
X.2.2 PACKET FORMAT -- CuSeeMePacket considerations
As mentioned above, Aux Data headers and data currently occupy their own network packets, the very front of which contains a "CuSeeMePacketHeader". For Aux Data packets, that header has the following values in its "vHeader.dataType" field:
kAuxDataTypeMin (256) kAuxDataTypeMax (kAuxDataTypeMin+2-1) kAuxDataTypeSupervisor (kAuxDataTypeMin) kAuxDataTypeData (kAuxDataTypeMin+1)This provides for the two major categories of Aux Data packets: "supervisory" and "application data"; within this range of types, the supervisory packets come first; it is possible that some other types may be defined in the future. Note that these values are in no way related to the "auxType" values used by Aux Data applications; these values are only used at the CU-SeeMe "system" level and by reflectors that might want to prune back Aux Data data packets.
Originally we had anticipated the need for a whole range of Aux Data packet types, and therefore we defined the "min" and "max" values. In addition, we had anticipated using the low-order 8 bits to encode what is now the 32-bit "type" value.
A potential awkwardness is the fact that the CuSeeMePacketHeader is not an exact multiple of 4 bytes in length. Since the Aux Data header and data follow immediately after the CuSeeMePacketHeader, the fields in the Aux Data header are not aligned on 4-byte boundaries, as might have been expected.
X.2.3 PACKET FORMAT -- Coding considerations
All of this material is given in the source files "Aux_Data_Transport.h" and "Aux_Data_Internal.h".
X.3.1 PACKET FORMAT -- Supervisor messages
At present, Aux Data supervisor messages are sent in their own packets, but they might someday be packed into unused portions of audio or video packets. To make that change simpler, each Aux Data supervisor message has its own special header; at present, this header includes the standard "CuSeeMePacketHeader". The "AuxDataSupervisorPacket" consists of three parts, the standard "CuSeeMePacketHeader", immediately followed by an "AuxDataSupervisorHeader", followed by additional data for the particular type of supervisory data. The "AuxDataSupervisorHeader" is as follows:
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | Version | Length | Type | Count |
+--------------+--------------+--------------+--------------+
(Note that this is shown as it appears on the network, reading from left to right and from top to bottom; this is in "Big Endian" format as used on 680x0 Macintoshes.)
The various fields are used as follows:
00h - Version (1 byte - char)
adshVersion: This field shows the version of this packet format and can be used to support enhancements to the protocol (or at least to prevent crashes caused by obsolete formats.)
01h - Length (1 byte - char)
adshHeaderLength: This byte contains the length of the header portion of the packet. That is, the "data" portion is this many bytes past the start of the header. This can help reduce compatibility problems as the protocol develops.
02h - Type (1 byte - char)
adshType: This specifies which type of supervisor data follows the header. The currently defined types are given below.
ASP_TYPE_HOLE_REPORT (1) ASP_TYPE_ITEM_LIST (2) ASP_TYPE_ITEM_ACK (3)03h - Count (1 byte - char)
adshCount: This contains the number of "things" that follow in the data area. Typically, a supervisor packet contains several instances of some control block, and so this count value is provided for convenience.
X.3.2 PACKET FORMAT -- Supervisor (Hole Report)
The Hole Report is sent out by a recipient when it detects a hole in an incoming item (except for the to-be-expected hole between the latest segment and the end of the item). The report is for a specific item of a specific type from a specific recipient but it can contain information about many holes, the number of which is contained in "adshCount".
The first few fields describe the item and its type and start at the beginning of the data area:
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | ahrItemNumber |
+--------------+--------------+--------------+--------------+
04h | ahrType |
+--------------+--------------+--------------+--------------+
08h | ahrUNUSED1 |
+--------------+--------------+--------------+--------------+
0Ch | ahrUNUSED2 |
+--------------+--------------+--------------+--------------+
00h - Item Number (4 bytes - long)
The "ahrItemNumber" field contains the same value as the Item Number field in the Aux Data header and is used to identify a particular item that is still available.
04h - Type (4 bytes - unsigned char array, but processed as an unsigned long)
The "ahrItemType" field contains the same value as the auxType field in the Aux Data header and, in effect, specifies the recipient Aux Data application.
08h - UNUSED (4 bytes - long)
This field is unused at present.
0Ch - UNUSED (4 bytes - long)
This field is unused at present.
Those fields are followed immediately by "adshCount" instances of the "ahrHoleEntry" structure, which looks like this:
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | aohOffset |
+--------------+--------------+--------------+--------------+
04h | aohLength |
+--------------+--------------+--------------+--------------+
08h | UNUSED |
+--------------+--------------+--------------+--------------+
00h -Offset (4 bytes - long)
The "aohOffset" field contains displacement from the beginning of the item to the beginning of this hole.
04h - Length (4 bytes - unsigned char array, but processed as an unsigned long)
The "aohLength" field contains the length of this hole.
08h - UNUSED (4 bytes - long)
This field is unused at present.
If some intermediary (such as a reflector) is able to satisfy the hole request by sending out a copy of the data for that hole, then that intermediary will convert the "aohLength" value into its 2's compliment. That shows the original sender (and any other intermediaries) that the hole has been filled, but it also allows the sender to keep statistics on network error rates (if it so desires).
When the hole report is received by the original sender, it will replace any previous report from that receiver for that item. When the sender finishes its current work for the item, it will then merge all of the pending hole reports from all reporting receivers and will then begin resending the data for the reported holes. If a hole is reported by only one receiver, then the sender will send directly; otherwise, it will broadcast the resends. During the process of satisfying the merged hole reports, the sender will continue to accumulate newer hole reports. With luck, they will all finally become satisfied and the item will be marked "ADS_DONE".
A recipient must be willing and able to accept duplicates of portions of items being received. There is no guarantee that a hole will be filled in any particular order, and so it is possible for a large hole to become two smaller holes; this can happen, for example, if a re-try packet gets lost.
X.3.4 PACKET FORMAT -- Supervisor (Item List)
The Available Item Report is sent out periodically by each sender to announce the continued availability of its items. This report can be used by a recipient in several ways: (1) if an in-progress item is not on the availability list, then the recipient can know that the item has been cancelled; (2) if an available item was never seen by the recipient, then it can request a resend of the whole item.
The following "AuxDataAvailableItemReport" structure is repeated as specified in "adshCount":
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | aarItemType |
+--------------+--------------+--------------+--------------+
04h | aarItemNumber |
+--------------+--------------+--------------+--------------+
08h | aarItemLength |
+--------------+--------------+--------------+--------------+
0Ch | aarItemState | aarItemPercent |
+--------------+--------------+--------------+--------------+
00h - Type (4 bytes - unsigned char array, but processed as an unsigned long)
The "aarItemType" field contains the same value as the auxType field in the Aux Data header and, in effect, specifies the recipient Aux Data application.
04h - Item Number (4 bytes - long)
The "aarItemNumber" field contains the same value as the Item Number field in the Aux Data header and is used to identify a particular item that is still available.
08h - Item Length (4 bytes - long)
The "aarItemLength" field is used by a recipient to limit its retry requests if the recipient is just joining a conference; it might impose too heavy a network load for large available items to be resent.
0Ch - Item State (2 bytes - short)
The "aarItemState" field contains the same values as the "adsState" field that is given to the item's sending application. It can be used to differentiate between items that are in progress and items that are complete but still available.
0Eh - Item Percentage (2 bytes - short)
The "aarItemPercentage" field shows how complete the item is.
(IMPORTANT NOTE FOR THE FUTURE: It is *possible* that this report will be expanded to include information about the magic "pruning" bits that are associated by each sender with its available aux data types. This has not yet been designed.)
X.3.5 PACKET FORMAT -- Supervisor (Item Ack)
The Item Ack is sent by a recipient when it has completely receive an item that was sent "with acknowledgment". The "AuxDataItemAck" structure is the only thing in the packet, and so the "adshCount" field is 1:
0 8 16 24 31
+--------------+--------------+--------------+--------------+
00h | aiaType |
+--------------+--------------+--------------+--------------+
04h | aiaItemNumber |
+--------------+--------------+--------------+--------------+
00h - Type (4 bytes - unsigned char array, but processed as an unsigned long)
The "aiaItemType" field contains the same value as the auxType field in the Aux Data header and, in effect, specifies the recipient Aux Data application.
04h - Item Number (4 bytes - long)
The "aiaItemNumber" field contains the same value as the Item Number field in the Aux Data header and is used to identify the item whose reception is now complete.
X.3.7 PACKET FORMAT -- Coding considerations All of this material is given in the source files "Aux_Data_Transport.h" and "Aux_Data_Internal.h".
X.4 ERROR RECOVERY
Aux Data error recovery depends upon several "overlapping" facilities, which may be generally described as follows. (Note that this assumes the Aux Data was sent via the normal "SEND" parameter and not via "SEND_WITH_ACK".)
Each item is sent in as many network packets as are necessary to contain the data. Since each packet contains a complete description of the item, any one packet is sufficient evidence to a recipient that the item exists; that is, as long as the recipient correctly receives even one packet from the item, then the recipient can know to request the missing portions of the item. As the packets arrive, the recipient "fills" in the holes in the item, and whenever a new hole is created (as the result of a packet's failure to arrive), the recipient sends back a "hole report". Back at the sender, when the item has been completely sent, the sending Aux Data Transporter examines all hole reports received so far for the item and proceeds to re-send the data to fill those holes. This process continues as long as the data remains available for sending.
Since a very small item might fit within a single network packet, and since such a small item might therefore be more likely to be completely lost than a larger, multi-packet, item, the Aux Data Transporter will send out 2 additional copies of any single-packet item. This is an arbitrary choice, of course. A recipient will ignore any data from an item that has already been completely received.
In case all of an item's packets fail to arrive, the sender also periodically transmits "availability reports" to show which items are available for sending. If a recipient sees that an item is available but the recipient has never received any data for the item, then it sends a "big hole report" showing that the entire item should be re-sent.
Critical to this whole approach to error recovery is the continued availability of the data, even after it has been originally sent. If the Aux Data Application withdraws the data too soon, then the sending Transporter cannot honor requests to fill holes. An application much strike a reasonable balance between keeping items around forever, thereby consuming memory, and withdrawing them too quickly, thereby defeating the error recovery mechanisms.