RTP Benchmarks

This repository contains all benchmarking code related to the benchmark of uvgRTP against LIVE555 and FFmpeg.

Directories uvgrtp, ffmpeg, and live555 contain the C++ implementations for RTP (latency) senders and receivers.

Script benchmark.pl can be used to automate the benchmark runs and its usage is described below.

Script parse.pl can be used to parse the output of benchmark runs.

This repository also contains file udperf.c which can be used to test the throughput of a network and it was used for the paper to determine the upper limit for UDP traffic using 1500-byte Ethernet frames.

Running the benchmarks

Example 1

Benchmark uvgRTP's send goodput. Run the benchmark configuration with 8 different thread settings so first starting with 8 threads, then 7 threads then 6 etc.

Each thread configuration will test all FPS values between the range 30 - 480 and and each FPS is tested 20 times. FPS is doubled so the tested values are: 30, 60, 120, 480

Each FPS value for each thread configuration provides one log file

Sender

./benchmark.pl \
   --lib uvgrtp \
   --role send \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 480 \
   --iter 20

Receiver

./benchmark.pl \
   --lib uvgrtp \
   --role recv \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 480 \
   --iter 20

Example 2

Benchmark uvgRTP's send goodput using netcat

Sender

./benchmark.pl \
   --lib uvgrtp \
   --role send \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 60 \

Receiver

./benchmark.pl \
   --lib uvgrtp \
   --role recv \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 60 \

Parsing the benchmark results

If the log file matches the pattern .*(send|recv).*(\d+)threads.*(\d+)fps.*(\d+)iter.* you don't have to provide --role --threads or --iter

Parsing one output file

Parse one benchmark, generic file name

./parse.pl \
   --lib ffmpeg \
   --role recv \
   --path log_file \
   --threads 3
   --iter 20

Parse one benchmark, output file generated by benchmark.pl

./parse.pl --path results/uvgrtp/send_results_4threads_240fps_10iter

Parsing multiple output files

NB: path must point to a directory!

NB2: --iter needn't be provided if file names follow the pattern defined above

Find best configuration

Find the best configurations for maximizing single-thread performance and total performance where frame loss is less than 2%

./parse.pl --path results/uvgrtp/all --iter 10 --parse=best --frame-loss=2 

Output goodput/frame loss values to a CSV file

./parse.pl --path results/uvgrtp/all --parse=csv