uvgrtp-base/benchmarks

Benchmarks

Send benchmark is the coordinator in the benchmarks so it must be started first.

Running the benchmarks

Example 1

Benchmark uvgRTP's send goodput. Run the benchmark configuration with 8 different thread settings so first starting with 8 threads, then 7 threads then 6 etc.

Each thread configuration will test all FPS values between the range 30 - 480 and and each FPS is tested 20 times. FPS is doubled so the tested values are: 30, 60, 120, 480

Each FPS value for each thread configuration provides one log file

Sender

./benchmark.pl \
   --lib uvgrtp \
   --role send \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 480 \
   --iter 20

Receiver

./benchmark.pl \
   --lib uvgrtp \
   --role recv \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 480 \
   --iter 20

Example 2

Benchmark uvgRTP's send goodput using netcat

Sender

./benchmark.pl \
   --lib uvgrtp \
   --role send \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 60 \

Receiver

./benchmark.pl \
   --lib uvgrtp \
   --role recv \
   --use-nc \
   --addr 127.0.0.1 \
   --port 9999 \
   --threads 3 \
   --start 30 \
   --end 60 \

Parsing the benchmark results

If the log file matches the pattern .*(send|recv).*(\d+)threads.*(\d+)fps.*(\d+)iter.* you don't have to provide --role --threads or --iter

Parsing one output file

Parse one benchmark, generic file name

./parse.pl \
   --lib ffmpeg \
   --role recv \
   --path log_file \
   --threads 3
   --iter 20

Parse one benchmark, output file generated by benchmark.pl

./parse.pl --path results/uvgrtp/send_results_4threads_240fps_10iter

Parsing multiple output files

NB: path must point to a directory!

NB2: --iter needn't be provided if file names follow the pattern defined above

Find best configuration

Find the best configurations for maximizing single-thread performance and total performance where frame loss is less than 2%

./parse.pl --path results/uvgrtp/all --iter 10 --parse=best --frame-loss=2 

Output goodput/frame loss values to a CSV file

./parse.pl --path results/uvgrtp/all --parse=csv