Reassembler Component Type and Subtype

Packetizers have a component type of kRTPReassemblerType ('rtpr'). The subtype can be any four-character combination. Note, however, that all-lowercase types are reserved by Apple.

The 'rsmi' Public Resource

A reassembler must provide a public resource of type 'rsmi'. The public resource contains information about the capabilities of a given reassembler. This information lists the RTP payload types the reassembler can work with. In addition, it provides information about the reassembler’s performance characteristics, specifically its speed and ability to recover from lost packets.

If more than one reassembler is available for a given RTP payload type, QuickTime will choose the one with the best performance characteristics, such as speed or ability to deal with packet loss.

The format of the public resource is defined in QTStreamingComponents.r as follows:

type 'rsmi' {

    array infoArray {

        align long;

        longint = $$CountOf(characteristicArray); /* Array size */

        array characteristicArray {

            hex longinttag;

            hex longint value;

        };

hex longintpayloadFlags;

    /* kRTPPayloadTypeStaticFlag or kRTPPayloadTypeDynamicFlag */

        byte    payloadID;      /* if static payload */

        byte = 0;

        byte = 0;

        byte = 0;

        cstring;                    /* if dynamic payload */ 

    };

#define kRTPPayloadSpeedTag 'sped'/* 0-255, 255 is fastest */ ]

#define kRTPPayloadLossRecoveryTag 'loss'

    /* 0-255, 0 can't handle any loss, 128 can handle 50% packet loss */

#define kRTPPayloadTypeStaticFlag 0x00000001

#define kRTPPayloadTypeDynamicFlag 0x00000002

The payload flags field is set to kRTPMPPayloadTypeDynamicFlag if the reassembler handles a dynamic payload type, or kRTPMPPayloadTypeStaticFlag if it handles a static type.

The payload ID field of the 'rsmi' resource is set to the IETF-defined RTP payload value if a static payload type is used.

The C string contains the RTP payload type text for dynamic types.

A declaration in a .r file might look like this:

resource kRTPReassemblerInfoResType (128) {

    {

        {

            kRTPPayloadSpeedTag, 128,

            kRTPPayloadLossRecoveryTag, 50

        },

        kRTPPayloadTypeDynamicFlag, 0, "x-oval"

    }

};

This resource indicates that the reassembler is of average speed and that it can handle 50% packet loss while still providing some meaningful data. It handles the dynamic RTP payload type identified by x-oval.

The speed tag is relative to other reassemblers of the same type. A value of 128 is a reasonable default.

The Base Reassembler

Your packet reassembler relies on a base reassembler component to do most of the routine reassembly work. Your reassembler can modify some of the base reassembler’s default behaviors simply by setting flags. With a few exceptions, your reassembler implements reassembler functions only when it needs to override the normal behavior of the base reassembler.

The base reassembler’s main function is to assemble incoming packets into “chunks”, then pass the chunks to a stream handler. A chunk is the amount of data useful to a particular stream handler, such as a video frame or twenty milliseconds of audio.

Your reassembler must specify the type of stream handler needed, but your reassembler doesn’t work with stream handlers directly. The base reassembler does this for you.

The default behavior of the base reassembler is as follows:

1. The base reassembler fills out an RTPRssmPacket struct for each packet it receives.

2. The base reassembler puts the packets into a packet list, which corresponds to a data chunk. By default, the packet list contains all the packets with the same RTP transmission time, which is also the sample time. The base reassembler will start a new packet list when it receives a packet with a different RTP transmission time or with the RTP marker bit set. Your reassembler can change this default by telling the base reassembler that every packet is a chunk.

3. Once a packet list is complete, the base reassembler creates a chunk from it. By default, the chunk is made by concatenating the payload of each packet in the packet list. By default, the payload is assumed to be the entire contents of the packet following the RTP header and the payload header. The base reassembler assumes a zero-length payload header (no payload header) unless your reassembler sets a nonzero payload header length as the default.

4. By default, the base reassembler discards chunks with missing packets. The base reassembler can be set to inform your reassembler that a chunk has missing packets.

By implementing certain functions, you can cause the base reassembler to call your reassembler at one of several points to override or modify the default behavior:

This is discussed further in the sections Handling Packets Yourself and Handling Chunks Yourself.

Opening Your Reassembler

QuickTime may open your packet reassembler to check its version or to get information. It may then close your reassembler without ever initializing it or using it to process packets.

Your reassembler component must be able to perform the standard component functions, such as _Version, and its specific RTPRssmGetInfo function, without being initialized.

When opened, your packet reassembler must open a base reassembler component. Your reassembler should delegate any functions it does not implement. It should also support the _Target call.

A typical reassembler’s _Open and _Target functions might look like this:

pascal ComponentResult RTPOvalRssm_Open(RTPOvalRssmGlobalsPtr globals,

                                             ComponentInstance self) 

{

#pragma unused (inGlobals)

    ComponentResult err;

    globals = (RTPOvalRssmGlobalsPtr)

                              NewPtrClear(sizeof(RTPOvalRssmGlobalsRecord)); 

    if ( (err = MemError()) != noErr )

        goto exit;

    SetComponentInstanceStorage(self, (Handle)globals);

    globals->self = self;

    err = OpenADefaultComponent(kRTPReassemblerType, kRTPBaseReassemblerType,

                                &globals->delegateComponent); 

    if ( err == noErr )

        err = RTPOval_Target(globals, self);

exit:

    return err;

}

pascal ComponentResult RTPOval _Target(RTPOvalRssmGlobalsPtr inGlobals,

                                ComponentInstance inParentComponent) 

{

    inGlobals->parent = inParentComponent;

    return ComponentSetTarget(inGlobals->delegateComponent, inParentComponent);

}

Initialization

Your reassembler must implement the RTPRssmInitialize function. When initialized, your reassembler is passed an RTPRssmInitParams struct containing three initialization parameters:

struct RTPRssmInitParams {

    UInt32      reserved;

    UInt8       payloadType;

    UInt8       pad[3];

    TimeBase    timeBase;

    TimeScale   controlTimeScale;

};

typedef struct RTPRssmInitParamsRTPRssmInitParams;

During initialization, your reassembler should open a stream handler of the appropriate type. For example, if your reassembler works with H.261 packets, it should open a video stream handler. Stream handler types are specified in the same manner as track types (video is videoMediaType, and so on). There are currently stream handlers for audio, video, text, and MIDI.

Your reassembler opens a stream handler by calling RTPRssmNewStreamHandler. You must specify the stream handler type when you open it. If your reassembler handles multiple media types, it can open a stream handler later, after it learns what kind of media is in the stream.

You should initialize the sample description of the stream handler when you open it, if possible. It can be set or changed later if necessary. The stream handler will be unable to process any data until its sample description is set.

Your reassembler should also initialize the timescale of the stream handler at this time. If your reassembler needs to get the timescale from the stream, it can monitor incoming packets and set the timescale when it is known. (No chunks will be sent to the stream handler until its timescale is set).

During initialization, your reassembler should call RTPRssmSetCapabilities with any initial flags to control the base reassembler’s default behaviors, such as kRTPRssmEverySampleAChunkFlag or kRTPRssmTrackLostPacketsFlag.

You should also call SetPayloadHeaderLength at this time if you know the payload header length for your packets.

A typical reassembler initialization function might look like this:

EXTERN_API( ComponentResult )

RTPRssmOVAL_Initialize(

RTPHOVALGlobalsPtrinGlobals,

RTPRssmInitParams *inInitParams )

{

#pragma unused(inInitParams)

ComponentResulterr = noErr;

ImageDescriptionHandle imageDesc;

SInt32     flags = 0;

inGlobals->fTimeScale = kOVALRTPTimeScale;

flags = kRTPRssmQueueAndUseMarkerBitFlag + kRTPRssmTrackLostPacketsFlag;

err = RTPRssmSetCapabilities(inGlobals->delegateComponent, flags, -1L );

    if (err == noErr) {

        imageDesc = __GetMyImageDesc( inGlobals );

        if( imageDesc )

        {

            err = RTPRssmNewStreamHandler(inGlobals->delegateComponent,

              VideoMediaType, ( SampleDescriptionHandle ) imageDesc,

              inGlobals->fTimeScale, NULL);

        }  

        else

            err = memFullErr;

    }

    return err;

}

Setup and Information Functions

If you do not open a stream handler and set its timescale during initialization, you must implement the RTPRssmGetTimeScaleFromPacket function. If you have not set the stream handler’s timescale, the base reassembler will call your reassembler’s GetTimeScaleFromPacket function with every incoming packet until a stream handler is open and its time scale is set.

If you cannot determine the timescale based on the contents of the packet, return qtsUnknownValueErr or a 0 timescale.

EXTERN_API( ComponentResult ) RTPRssmGetTimeScaleFromPacket(

    RTPReassembler rtpr,

    QTSStreamBuffer *inStreamBuffer,

    TimeScale * outTimeScale);

The RTPRssmGetInfo, RTPRssmSetInfo, and RTPRssmHasCharacteristic functions can be called at any time, even prior to initialization. RTPRssmHasCharacteristic is called to determine what features your reassembler supports. RTPRssmGetInfo is used to get information from your reassembler. RTPRssmSetInfo could be used to send information to your reassembler, but there are currently no selectors that do this.

These functions are commonly used if your reassembler supports passing of non-media data, such as transformation matrices. The functions are typically called after your reassembler reports a change in a non-media parameter by calling RTPRssmSendStreamHandlerChange, as described in Passing Non-Media Data. The selectors used for passing non-media data are the same for RTPRssmHasCharacteristic and RTPRssmGetInfo. These selectors begin with kQTSSource and are defined in QTStreaming.h.

Delegate any selectors you do not support or do not understand to the base reassembler for all of these functions:

EXTERN_API( ComponentResult ) RTPRssmSetInfo(

    RTPReassembler    rtpr,

    OSType           inSelector,

    void *           ioParams) ;

EXTERN_API( ComponentResult ) RTPRssmGetInfo  (

    RTPReassembler rtpr,

    OSType     inSelector,

    void *     ioParams) ;

EXTERN_API( ComponentResult ) RTPRssmHasCharacteristic(

    RTPReassembler rtpr,

    OSType      inCharacteristic,

    Boolean *   outHasIt) ;

Handling Packets Yourself

Your packet reassembler can be called when each packet is received. If you have not yet opened a stream handler and set its timescale, you will receive a RTPRssmGetTimeScaleFromPacket call for each received packet.

Once the stream handler’s timescale is set, the default behavior for the base reassembler is to fill out an RTPRssmPacket struct for each incoming packet, then to add each packet to the current packet list. The base reassembler starts a new packet list when a packet’s RTP marker bit is set, or the RTP timestamp changes, or if your reassembler has set the kRTPRssmEveryPacketAChunkFlag in RTPRssmSetCapabilities.

If you want to fill out or modify the RTPRssmPacket struct for each packet yourself, because you use variable payload header lengths, for example, implement the RTPRssmAdjustPacketParams function.

EXTERN_API( ComponentResult ) RTPRssmAdjustPacketParams(

    RTPReassembler    rtpr,

    RTPRssmPacket     *inPacket,

    SInt32            inFlags) ;

The inPacket parameter points to the RTPRssmPacket struct for the current packet. This structure has already been filled in by the base reassembler:

struct RTPRssmPacket {

    struct RTPRssmPacket      *next;

    struct RTPRssmPacket      *prev;

    QTSStreamBuffer           *streamBuffer;

    Boolean                   paramsFilledIn;

    UInt8                     pad[1];

    UInt16                    sequenceNum;

    UInt32                    transportHeaderLength;

    UInt32                    payloadHeaderLength;

    UInt32                    dataLength;

    SHServerEditParameters    serverEditParams;

    TimeValue64               timeStamp;

    SInt32                    chunkFlags; 

    SInt32                    packetFlags;

}; typedef struct RTPRssmPacketRTPRssmPacket;

Handling Chunks Yourself

The base reassembler automatically builds chunks from packets and sends them to the stream handler you have opened. If any packets are missing from the chunk, the base reassembler discards the entire chunk.

You can take over from the base reassembler at several points in the chunk-building process by implementing the appropriate function. Do this if your reassembler performs loss recovery, or if the base reassembler’s default behavior needs to be modified to correctly build chunks for the stream handler.

Your reassembler is responsible for creating and sending the chunk, beginning at whatever point you take over, and continuing until the chunk is sent or discarded.

The process of creating a chunk begins when the base reassembler has a complete packet list. To take over at this point, implement RTPRssmSendPacketList. You might want to do this if you were altering the packet list for loss recovery.

If you implement RTPRssmSendPacketList, you are responsible for deleting the packet list when you are through with it, by calling RTPRssmReleasePacketList.

Next, the base reassembler calculates the chunk size. It does this by summing the payload size for each packet in the list. The payload size is calculated by subtracting the RTP header and the payload header from the packet size. To take over at this point, implement RTPRssmComputeChunkSize. You might want to do this if you need to add data to the chunk that isn’t in the packets, or to use a different method for calculating the payload size.

If you implement RTPRssmComputeChunkSize, you will need to allocate a chunk of the appropriate size. To do this, call RTPRssmGetChunkAndIncrRefCount. This will allocate the chunk and set the number of references to it to 1. QuickTime will maintain the chunk until the number of references to it is 0. The reference count is automatically decremented when you send the chunk. You can increment the counter by calling RTPRssmIncrChunkRefCount if you want QuickTime to preserve the chunk for later use (by substituting for a lost chunk, for example).

Once the chunk size has been calculated and the chunk has been allocated, the base reassembler moves the data from the packets into the chunk. The data moved will be a simple concatenation of the packet payloads. The payload size and offset within each packet are calculated in the same manner as used for calculating chunk size. To be able to take over at this point, implement RTPRssmCopyDataToChunk. You would need to do this to modify the bytes at packet boundaries for an H.261 packet reassembler, for example.

Bear in mind that if you implement RTPRssmSendPacketList, you are responsible for the steps performed in RTPRssmComputeChunkSize and RTPRssmCopyDataToChunk. Similarly, if you implement RTPRssmComputeChunkSize, you must perform the steps for RTPRssmCopyDataToChunk. Only the first of these functions that you implement will be called. You take over the chunk-building process from there.

If you implement any of the three functions just discussed, you are responsible for sending the chunk by calling RTPRssmSendChunkAndDecrCount.

Other things your reassembler might do at this point are to

• Tell the stream handler that the sample description has changed (RTPRssmSetSampleDescription).

• Tell the stream handler that some non-media data has changed, such as a transformation matrix or sound volume (RTPRssmSendStreamHandlerChanged ). You may receive a series of GetInfo calls to determine what has changed.

• Tell the stream handler not to send the chunk. The reference counter will be decremented; if it becomes zero, the chunk is deallocated (RTPRssmSendLostChunk).

• Tell the base reassembler to keep a copy of this chunk for your later use (RTPRssmIncrChunkRefCount). Be sure to increment the ref count before you send the chunk. You must eventually call RTPRssmDecrChunkRefCount for every call to RTPRssmIncrChunkRefCount or you will create a memory leak.

Reset and Clear Cache Functions

Your reassembler can be called at any time with the RTPRssmReset function, which you must implement. Reset all your variables, release any chunks being held for you, and prepare for a new run of data. At the end of an RTPRssmReset function, your state should be identical to its state after the first RTPRssmInitialize call.

You can instruct the base reassembler to release any packets in the packet list it is currently building by calling RTPRssmClearCachedPackets. You might do this if you are handling packets yourself and you determine that the list the base reassembler is building should be discarded.