StreamWorks Server Technical Reference
StreamWorks Server Technical Reference
StreamWorks Server Configuration File (``streamworks.cfg'')
The StreamWorks Server Process (SWserver)
MPEG Basics
StreamWorks Server Configuration File ("streamworks.cfg")
General Parameters
The following parameters appear in the "[General]" section of the streamworks.cfg file:
ActivationKey
- (required) your unique server activation key that controls server capabilities;
ServerRoot
- (optional; default=current directory) specifies the directory (as a full UNIX path) to which the StreamWorks Server process (SWserver) looks for its associated files.
StreamRoot
- (optional; default=ServerRoot) specifies the location as a UNIX path (relative to ServerRoot unless preceded by a forward slash "/") of the directory in which audio/video files are stored.
PlayRoot
- (optional; default=ServerRoot) specifies the location as a UNIX path (relative to ServerRoot unless preceded by a forward slash "/") of the directory in which PLAY files are stored.
User
- (optional; default=current user) specifies the owner of the StreamWorks Server process (SWserver); the specified userID is assumed only when the SWserver program is started by "root" (contact your host's administrator if you have questions regarding process ownership).
Group
- (optional; default=current usergroup) specifies the group ownership of the StreamWorks Server process (SWserver); the specified groupID is assumed only when the SWserver program is started by "root" (contact your host's administrator if you have questions regarding process ownership).
MaxBandwidth
- (optional; no default) specifies the maximum total data rate (in bits per second) of all simultaneously served streams. This parameter effectively limits the number of simultaneous users your server supports.
Port
- (required; default=1558, Xing's registered port) specifies the port to which the StreamWorks Server listens for stream requests.
Logging Parameters
The following parameters appear in the "[Logging]" section of the streamworks.cfg file:
PIDLog
- (optional; no default) specifies the location as a UNIX pathname (relative to ServerRoot unless preceded by a forward slash "/") of the file to which the processID of the StreamWorks Server process (SWserver) is written.
ErrorLog
- (optional; no default) specifies the location as a UNIX pathname (relative to ServerRoot unless preceded by a forward slash "/") of the file to which the StreamWorks Server process (SWserver) writes error codes.
ConnectionLog
- (optional; no default) specifies the location as a UNIX pathname (relative to ServerRoot unless preceded by a forward slash "/") of the file to which the StreamWorks Server process (SWserver) writes connection data.
The remaining entries in the [Logging] section control the contents of the ConnectionLog:
IPAddress
- (optional; default=False) specifies if ConnectionLog entries include the IPaddress of the device to which the stream is transmitted.
Domain
- (optional; default=False) specifies if ConnectionLog entries include the Domain of the device to which the stream is transmitted.
StreamID
- (optional; default=False) specifies if ConnectionLog entries include the stream specification (i.e., server:port/stream) requested by the StreamWorks Player.
UserID
- (optional; default=False) specifies if ConnectionLog entries include the unique identifier of the StreamWorks Player to which data is transmitted.
StartTime
- (optional; default=False) specifies if ConnectionLog entries include the date and time at which stream transmission started.
StopTime
- (optional; default=False) specifies if ConnectionLog entries include the date and time at which stream transmission stopped.
Duration
- (optional; default=False) specifies if ConnectionLog entries include the duration (calculated as the difference between StartTime and StopTime) of stream transmission.
StopMethod
- (optional; default=False) specifies if ConnectionLog entries include the method (e.g., user requested stop, network timeout, server error) by which stream transmission was stopped.
ClientType
- (optional; default=False) specifies if ConnectionLog entries include the platform and type (e.g., Mac, Helper) of the end-user's StreamWorks Player.
ClientVersion
- (optional; default=False) specifies if ConnectionLog entries include the version of the end-user's StreamWorks Player.
ConnectionSpeed
- (optional; default=False) specifies if ConnectionLog entries include the data rate (in kilobits per second, where kilo=1000) requested by the StreamWorks Player.
AudioRate
- (optional; default=False) specifies if ConnectionLog entries include the data rate (in kilobits per second, where kilo=1000) of audio data sent to the StreamWorks Player.
VideoRate
- (optional; default=False) specifies if ConnectionLog entries include the data rate (in kilobits per second, where kilo=1000) of video data sent to the StreamWorks Player.
PacketLoss
- (optional; default=False) specifies if ConnectionLog entries include the amount of network data (as a percentage of the total transmitted data) lost during transmission to the StreamWorks Player.
FrameLoss
- (optional; default=False) specifies if ConnectionLog entries include the amount of video/audio frames (as a percentage of the total frames received) lost during playback by the StreamWorks Player.
The StreamWorks Server Process (SWserver)
The StreamWorks Server Process (SWserver) is started when you run the
SWserver
program located in your "streamworks" directory. The SWserver process, and other (child) processes that it starts, provide all the services of your StreamWorks Server.
Each time an end-user of the StreamWorks Player requests a stream, the SWserver process starts new (child) processes:
each on-demand file consumes 2 processes for each user;
each live feed consumes 2 processes for all users at a given transmit rate;
MPEG Basics
The following article introduces the MPEG compression standard within the context of Xing's XingCD encoding software.
The MPEG Compression Algorithm
MPEG is the popular name for ISO/IEC DIS 11172, the video compression standard developed under the auspices of the International Standards Organization (ISO) by the Moving Picture Experts Group. MPEG-1, the first algorithm to be completed, covers compressed video and audio at data rates up to 1.8 Mbits/sec. The primary design goal for MPEG-1 was SVHS quality video playback from CD-ROM. The committee is now finalizing an extension of the first algorithm, called MPEG-2, which addresses higher resolutions and interlacing for broadcast TV and HDTV. Meanwhile MPEG ad hoc groups monitor research in other areas of video compression, such as extremely low bit rate coding for telecommunication applications.
Like the JPEG still image compression standard, MPEG is a multi-stage algorithm built around the discrete cosine transform (DCT). The two algorithms also share similar approaches to color space conversion, blocking, quantization, zigzag ordering, run length tokenizing, and Huffman coding. But MPEG goes farther, adding interframe compression and interleaved audio. As with most interframe compression schemes, MPEG is asymmetric--compression requires more effort than decompression.
To better illustrate MPEG's inner workings, let's walk through an example XingCD compression job. Starting with 640x480x24 bit Targa files (one file per frame as an animation program or capture board might produce), XingCD will produce a single MPEG file ready for CD-ROM mastering.
First XingCD scales the original frames to the desired resolution. MPEG allows for arbitrary frame sizes, but most MPEG-1 video is compressed at the Source Input Format (SIF) resolution: 352x240x30 fps for NTSC and 352x288x25 fps for PAL. Before compressing, XingCD scales source video frames to the desired output resolution whether it be SIF or a user-defined resolution. XingCD now deals with color space. MPEG specifies the YCbCr color space instead of the more familiar RGB in Targa files. The luminance component, Y, represents brightness. The chrominance components, Cb and Cr, are color differences. In the TV world, "black and white" sets process only luminance, while color sets contain additional circuitry to handle chrominance. XingCD accomplishes color space conversion with a 3x3 matrix multiply:
Y = 0.299R + 0.587G + 0.114B Cb = - 0.1687R - 0.3313G + 0.5B Cr = 0.5R - 0.4187G - 0.0813B
YCbCr permits more efficient compression. Psychovisual tests indicate that the human eye is more sensitive to detail in the luminance component. XingCD takes advantage of this feature, subsampling the chrominance pixels by half horizontally and vertically. XingCD no has a 352x240 Y plane, a 176x120 Cb plane, and a 176x120 Cr plane. This cuts frame storage requirements by half, from 253440 to 126720 bytes, but retains the ability to display 16.7 million colors. Next XingCD divides the image into blocks and macroblocks. Each 8x8 block contains 64 Y, Cb, or Cr pixels. Because of subsampling, the Cb and Cr blocks cover the same area as four Y blocks (see figure). The resulting group of six blocks (4 Y, 1 Cb and 1 Cr) is called a macroblock.
At this point the path branches depending on which frame type XingCD is producing. MPEG uses three frame types: Intra, Predicted, and Bidirectional. Intra, or "I" frames, are similar to JPEG still images. Each macroblock is coded without any reference to other frames. Predicted, or "P" frames, refer to a previous I or P reference frame. Bidirectional, or "B" frames, refer to both a previous and future I or P reference frame. XingCD lets you control the distribution of I, P, and B frames to meet specific needs (e.g. including more I frames to improve random access performance) by specifying Group and Subgroup sizes. Group sizes determine the frequency of I frames (e.g., group size = 15 causes an I frame to occur every 15 frames), while subgroup sizes determine the frequency of reference frames (e.g., subgroup size = 5 causes an I or P frame to occur every 5 frames).
If the current frame is to be an I frame, which does not depend on any other frame, XingCD proceeds on to the next step. If it is a P frame, XingCD first performs interframe correlation. For each macroblock XingCD searches the reference frame looking for the best match. If XingCD finds an exact or near-exact match it needs to encode a pointer to the matching pixels. This pointer, called a motion vector, typically requires only a few bits of storage space. Coding an entire 16x16 macroblock in less than a single byte is extraordinary compression indeed! The downside of this efficiency is the computational cost of the search, which is the most time consuming task an MPEG encoder performs.
Often a perfect match is unavailable because objects don't simply float across the screen; they also rotate, fade, change shape, and move toward or away from the viewer. In these cases the search may still turn up a partial match. XingCD then computes the difference between the current pixels and the reference pixels. The resulting difference pixels are often highly correlated and therefore amenable to compression. Blocks coded differentially require more storage than perfectly matched blocks, but differential coding still saves bytes over intra coding. In the rare case of search results so poor that even differential coding is not practical, XingCD simply codes that macroblock as intra-type.
The primary change to this approach for B frames is that XingCD searches both a previous and a future frame for reference pixels. Using two separate reference frames yields higher correlation and therefore better compression. Consider a traffic video with cars entering the field of view from the left and leaving it on the right. When a car first appears, it will be impossible to find good pixel matches in the previous reference frame. In this case, the encoder will select reference pixels from the future reference frame. The situation is reversed as the car leaves the field of view. The ability to choose between reference frames is a powerful encoding tool. Even better, when matches are found in both reference frames the encoder can average the two together to reduce noise and further improve correlation.
After interframe correlation, blocks being coded as intra-type and differentially encoded P/B blocks are fed into a discrete cosine transform (DCT). The DCT, a once-obscure linear transform which now seems to be on the tip of everyone's tongue, maps an 8x8 block of pixels into 64 frequency coefficients. Each coefficient represents a weighting factor for a corresponding cosine curve. The 64 basis cosine curves vary in frequency--low frequencies describe the block's coarse structure while high frequencies fill in the detail. Adding the 64 weighted basis curves together will reproduce the original 64 pixels. By itself, the DCT provides no compression. But the lack of extreme detail in most image blocks means high frequency coefficients are typically zero or near zero.
To increase the number of zero frequencies and reduce the number of bits needed for non-zero frequencies, XingCD divides each coefficient by a quantizer. Since the eye is less sensitive to errors in high frequency coefficients, the quantizer values tend to increase with frequency. MPEG uses one quantizer table for intra macroblocks and another for non-intra. XingCD can use the default quantizer tables listed in the MPEG spec or devise custom tables.
Quantizing causes a loss of image content. Large quantizer values cause more loss (i.e. greater image degradation) but also deliver higher compression. XingCD varies a quantizer scale factor to maintain the constant data rate specified by the user. If a frame is using more bits than allocated, XingCD increases the quantizers until the bit rate falls back into line. Conversely, if the bit rate drops below the threshold XingCD can decrease the quantizers and improve image quality.
The quantized frequencies are zigzag ordered. This step produces long zero runs ideally suited to run length coding. In run length coding XingCD expresses each zero run as a token describing the number of zero frequencies in the run and the value of the non-zero frequency which ends it. These tokens are further compressed through Huffman coding, which converts each token into a variable length code (VLC). VLCs for the most common tokens are 2-3 bits long, whereas VLCs for rare tokens are up to 28 bits long. The final bitstream, consisting mainly of very short codes, is roughly one third the size of the run token stream.
As XingCD writes compressed data to a file, it inserts MPEG-defined headers to assist the decoder. Each header begins with a unique 32 bit start code. A sequence header at the beginning of the file defines the frame rate, resolution, etc. A group of pictures header appears before I frames to indicate random access entry points. Picture headers identify each frame and communicate frame specific information such as I, P, or B frame type. Finally, XingCD inserts slice codes at various locations in the frame to give decoders a chance to re-synchronize if they detect an error condition (e.g. corrupted bitstream). The compressed data combined with the headers make up a fully defined MPEG video file.
Unless you are producing a silent movie, you'll also want sound. The MPEG audio compression algorithm uses subband filters to divide the audio signal into frequency bins. XingCD uses a sophisticated psycho-acoustic model to allocate bits among the different frequency bins. By allocating fewer bits to the less audible bins, XingCD can store CD quality sound at compression ratios up to 8:1. If you can live with a little audio degradation, XingCD can push the compression ratio as high as 20:1.
As a final step, XingCD breaks the compressed audio and video bitstreams into packets and interleave the packets into an MPEG System stream. During this process, XingCD inserts time stamps to help the decoder separate and synchronize the audio and video streams.