1.0 Assignment Overview

• Client code that receives streaming audio packets; implements basic error concealment; and plays the audio to an external speaker.

• An analysis of the effectiveness of various concealment techniques.

In the first part of the assignment you will write a client and a server to send and receive audio packets. However, you will not need to turn in your server, just the client. To help you test your client code in a way identical to how we will test your client when grading the assignment, we will provide you with an executable version of a server at some point before the assignment is due. Setting up the assignment this way achieves a secondary objective: understanding that writing a client and server is much more difficult when you are developing software to implement a written specification!

2.0 Assignment Background

For this type of real-time operation, and for this assignment, you can assume that if the jitter causes a packet to arrive after its playback time, then the packet is lost. Since the probability of losing a packet is non-zero, the receiver must have some policy for handling playout when some packets arrive after their playback time. You will experiment with some of these different policies.

Packets can also be lost because they arrive too quickly from the transmitter; the receiver's buffer fills up; and packets must be dropped. One way to solve this problem is to create additional space in the buffer by further compressing the audio data already in the buffer. You will experiment with different mechanisms for compressing mu-law encoded data.

3.0 Assignment Details

3.1 Streaming Audio Client

csil-machine> client <server_name_or_ip_address> <port_number> <loss_recovery_technique>


Please enter arguments:
<file_name> <packet_length> <loss_percentage>


Sending request...
Receiving and playing packets...
Done.


csil-machine>

Arguments:

• <server_name_or_ip_address>: Machine name or IP address.
• <port_number>: Port number.
• <loss_recovery_technique>: Either "SILENCE" or "LAST"
• <file_name>: File name without extension, e.g. "airwolf", "voice", etc.
• <packet_length>: Length of data portion of each packet (in bytes).
• <loss_percentage>: Express as a a decimal, e.g. 0.50, 0.75, etc.

csil-machine> client muddy.cs.ucsb.edu 32000 LAST


Please enter arguments:
simple 500 0.05


Sending request...
Receiving and playing packets...
Done.


csil-machine>

Other notes:

• The initial request is sent as a text string using UDP.

• The initial request should be of the form:
  <IP_address> <port_number> <file_name> <packet_length> <loss_percentage>.
  For example: 128.111.52.10 32000 voice 500 0.05

• When receiving the first packet, add in an initial buffer delay of about 20 ms to handle jitter cases.

• The client will know there are no more samples when it receives a null character. A null character should be sent by itself as the last packet. This last packet should always be sent.

• The server will deliver a binary data stream as "unsigned integers". The inter-packet transmission time will be computed by the server based on the packet length.

• You will not have to deal with error conditions, e.g. a request for a file name that does not exist, out of bound errors on the loss percentage, etc.

3.2 Analysis of Effects of Packet Loss

3.2.1 Simulation Model

main()
{
   open_audio_socket();
   open_audio_source_file();
   while not EOF {
      read_sample();   /* Read audio sample from a file */
      play_sample();   /* Play out the received sample */
   }
}

3.2.2 Writing Your Simulator

The second hardest part of this assignment is dealing with the audio format. You can make your life hard by choosing a complicated encoding and compression algorithm. You can make your life easy by choosing mu-law encoding which is the basic, uncompressed format that Sun uses in its *.au files. The challenge here is to find tools that can read and play these files.

To help you get through some of these problems, there is some C code in Section 3.2.2.1 to help you with device and file processing, and a location of some *.au files in Section 3.2.2.2.

3.2.2.1 Device/File Processing

To open the audio socket, use the following statement:

audio_device_ID = open("/dev/audio",O_WRONLY | O_NDELAY);

To open a file containing sample data, use the following statement:

source_file_ID = open("file1.au",O_RDONLY | O_NDELAY);

The return value of the ``open'' function is an integer that serves as the identifier for the opened device. For more information, do a ``man open''. Once the device or file is open, you can read or write using the ``read'' and ``write'' commands. Both statements have the same format, and look like the following:

return_code = read(device_identifier,buffer_variable,length);

The ``device_identifier'' is what is returned from an open command. The ``buffer_variable'' is the array used to store read data, or as a source for write data. The length specifies how long the data to be read or written is. For more information on the read or write statement, do a ``man 2v read'' or ``man 2v write''.

3.2.2.2 Sample Files

To help you get started, we have provided some sample audio files that you can use. These are located in the samples directory.

You can also play out any sample file, or record your own samples using the programs ``audiotool'', or ``soundtool''. In order to create your own sample files, you'll need a microphone.

3.2.2 Experiments

• Test the Effect of Packet Size. Each audio packet will contain some number of sample values. In some cases, a packet can contain hundreds or even thousands of samples. Furthermore, if you try to send only one packet to the audio device at a time, your workstation will probably not be able to deliver packets fast enough to playback the sound continuously. What is the minimum number of samples you must include in a packet in order to get continuous sound playout on your workstation? What would be a good number of samples to include in each packet? What happens when the packet size is increased further? Explain what you observe. (HINT: See the man pages for ``audio''.)

• Test the Effect of Loss of Packets. What happens as the network starts to lose packets, or packets are dropped because the receiver's buffer fills up? How is audio quality affected? Describe what happens when the percentage of lost packets increases from zero percent to greater than fifty percent. When does the loss become noticeable? When does the audio become unintelligible? How does the size of the packet affect the sound quality during lossy playback? (NOTE: The measurement of audio quality can be quite subjective. It is important to establish and use a qualitative scale that provides as much information as possible.)

• Policy for Playing Unavailable Packets. When a packet is lost, or does not arrive in time to be played, the receiver must have a policy for deciding what to do. Two policies include playing silence, or playing the last packet received. For this experiment, implement both solutions, and describe the difference in quality between the two. How is the sound quality different for the two solutions at low loss, and at high loss? Describe in some detail other policies that can be used to reduce the effects of lost packets.

• Silence Detection. Depending on the audio source, there may be long periods of silence. For example, while a song may not have many silence periods, a lecture or conversation may have many long silence periods. Using a silence detection threshold, each sample is either silence (if it is below the threshold) or sound (if it is above the threshold).

  For this experiment, you will use a silence detection threshold to pre-process and modify an audio stream. Any sample value that is below the threshold should be changed to zero. Pre-process different types of audio streams and compare the quality of the audio using a variety of silence detection thresholds. Remember to look at the mu-law encoding information to identify which encoded values represent silence or near silence.

• Silence Compression and Expansion. In this set of experiments, you are to examine the effect of shortening or lengthening the silence periods. Using the modified audio files from Step 4, pre-process the audio file again to create new files that have shorter and longer silence periods. Describe how sound quality is affected for different rates of compression and expansion. How much can you expand or compress an audio file? What happens when you add silence randomly to the audio file?

4.0 Deliverables

• Streaming Audio Client Code:
  • Turn in two files: client and client.* where the "*" is the extension representing the language used to write the client.

  • You should only turn in TWO files. No more, no less. Also, make sure the file names are exactly the same as given above.

  • The client program should be an executable version of your client code that runs on machines in the CSIL lab.

• Analysis of the Effects of Packet Loss:
  • Turn in a written writeup in class describing what tests your performed (be sure to describe parameters, factors, and metrics), what results you obtained, an analysis of the results, and conclusions. Results should be shown in tables or graphs and any factors should be tested over a sufficiently wide range of values as to allow a comprehensive analysis.