qiurui
/
uvgrtp-base
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

common: Separate fps enforcement and fragment pacing 

This commit also fixes (mostly) the fps enforcement.
This commit is contained in:
Joni Räsänen 2022-09-16 15:20:32 +03:00
parent 02c764a41c
commit 76b13879e5
7 changed files with 152 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
docs/README.md
View File

@ -84,6 +84,8 @@ session->create_stream(..., RCE_SRTP | RCE_SRTP_KMNGMNT_ZRTP | RCE_SRTP_NULL_CIP
| RCE_SRTP_KEYSIZE_192 | Use 196 bit SRTP keys, currently works only with RCE_SRTP_KMNGMNT_USER |
| RCE_SRTP_KEYSIZE_256 | Use 256 bit SRTP keys, currently works only with RCE_SRTP_KMNGMNT_USER |
| RCE_ZRTP_MULTISTREAM_NO_DH | Select which streams do not perform Diffie-Hellman with ZRTP. Currently, ZRTP only works reliably with one stream performing DH and one not performing it |
| RCE_FRAMERATE | Try to keep the sent framerate as constant as possible (default fps is 30) |
| RCE_FRAGMENT_PACING | Pace the sending of framents to frame interval to help receiver receive packets (default frame interval is 1/30) |
### RTP Context Configuration (RCC) flags
@ -103,8 +105,8 @@ stream->configure_ctx(RCC_PKT_MAX_DELAY, 150);
| RCC_DYN_PAYLOAD_TYPE | Override uvgRTP's default payload number used in RTP headers | Format-specific, see [util.hh](/include/uvgrtp/util.hh) | Both |
| RCC_CLOCK_RATE | Override uvgRTP's default clock rate used to calculate RTP timestamps | Format-specific, see [RFC 3551](https://www.rfc-editor.org/rfc/rfc3551#section-6) | Sender |
| RCC_MTU_SIZE | Set the Maximum Transmission Unit (MTU) value. uvgRTP assumes the repsence of UDP header (8 bytes) and IP header (20 bytes for IPv4) and substract those from the value. | 1500 bytes | Both |
| RCC_FPS_ENUMERATOR | Setting RCC_FPS_ENUMERATOR or RCC_FPS_DENOMINATOR enables the constant fps functionality. With this functionality, uvgRTP will send fragmented frames with correct transmission interval and divides each fragment for that frame interval to help receiver in receiving packets. Use this in challenging performance circumstances or to achieve extreme performance with uvgRTP. Adds one frame of latency. Disable with 0. | 30 (disabled) | Sender |
| RCC_FPS_DENOMINATOR | Use this in combination with RCC_FPS_ENUMERATOR if you need fractional fps values | 1 (disabled) | Sender |
| RCC_FPS_ENUMERATOR | Set the fps used with RCE_FRAMERATE and RCE_FRAGMENT_PACING. | 30 | Sender |
| RCC_FPS_DENOMINATOR | Use this in combination with RCC_FPS_ENUMERATOR if you need fractional fps values | 1 | Sender |
### RTP frame flags
@ -154,12 +156,12 @@ The second way of handling key-management of SRTP is to do it outside uvgRTP. To
The default MTU size of uvgRTP has been set to 1500. uvgRTP assumes the presence of an UDP header and IP header in addition an RTP header which are taken into account when fragmenting frames. If your application is expected to work through tunneling such as VPN which adds additional headers on top of packets, you should lower the MTU size to avoid unnecessary IP level fragmentation. Some networks also allow for a higher MTU size in which case you can increase this.
## High-performance scenario
## Trouble receiving burst of packets?
The default configuration of uvgRTP should be able to handle most basic scenarios up to 4K30p without any frame loss. If you are however 1) using a higher resolution, 2) higher fps value, 3) using a low power machine to receive the RTP stream, or 4) you are experiencing frame loss, you might consider setting or increasing the following parameters:
* RCC_UDP_RCV_BUF_SIZE: You can try increasing this to 40 or 80 MB if it helps receiving frames
* RCC_UDP_SND_BUF_SIZE_ You can try increasing this to 40 or 80 MB if it helps sending frames
* RCC_RING_BUFFER_SIZE: You can try increasing this to 40 or 80 MB if it helps receiving frames
* RCC_FPS_ENUMERATOR and RCC_FPS_DENOMINATOR: You can try setting these to stream fps value to help sending and reception of frames
* RCE_FRAGMENT_PACING, RCC_FPS_ENUMERATOR and RCC_FPS_DENOMINATOR: You can try RCE_FRAGMENT_PACING to make sender pace the sending of framents so receiver has easier time receiving them. Use RCC_FPS_ENUMERATOR and RCC_FPS_DENOMINATOR to set your fps
None of these parameters will however help if you are sending more data than the receiver can process, they only help when dealing with burst of (usually fragmented) RTP traffic.

8
include/uvgrtp/util.hh
View File

@ -272,7 +272,13 @@ enum RTP_CTX_ENABLE_FLAGS {
/** Select which ZRTP stream does not perform Diffie-Hellman negotiation */
RCE_ZRTP_MULTISTREAM_NO_DH = 1 << 17,
RCE_LAST = 1 << 18
/** Force uvgRTP to send packets at certain framerate (default 30 fps) */
RCE_FRAMERATE = 1 << 18,
/** Paces the sending of frame fragments within frame interval (default 1/30 s) */
RCE_FRAGMENT_PACING = 1 << 19,
RCE_LAST = 1 << 20
}; // maximum is 1 << 30 for int

40
src/frame_queue.cc
View File

@ -29,6 +29,7 @@ uvgrtp::frame_queue::frame_queue(std::shared_ptr<uvgrtp::socket> socket, std::sh
rtp_(rtp),
socket_(socket),
rce_flags_(rce_flags),
fps_(false),
frame_interval_(),
fps_sync_point_(),
frames_since_sync_(0)
@ -284,23 +285,39 @@ rtp_error_t uvgrtp::frame_queue::flush_queue()
if (active_->packets.size() > 1)
((uint8_t *)&active_->rtp_headers[active_->rtphdr_ptr - 1])[1] |= (1 << 7);
if (fps)
std::chrono::high_resolution_clock::time_point this_frame_time_slot = this_frame_time();
if ((rce_flags_ & RCE_FRAMERATE) && fps_)
{
std::chrono::high_resolution_clock::time_point this_moment = std::chrono::high_resolution_clock::now();
std::chrono::high_resolution_clock::time_point next_frame = next_frame_time();
if (next_frame > this_moment)
std::chrono::high_resolution_clock::time_point now = std::chrono::high_resolution_clock::now();
if (this_frame_time_slot < now || frames_since_sync_ == 0)
{
UVG_LOG_DEBUG("Updating fps sync point");
fps_sync_point_ = this_moment;
next_frame = next_frame_time();
UVG_LOG_DEBUG("Updating framerate sync point");
frames_since_sync_ = 0;
// have a little bit of extra room so we don't sync constantly
fps_sync_point_ = std::chrono::high_resolution_clock::now() + frame_interval_/10;
}
else if (this_frame_time_slot > now + std::chrono::milliseconds(1))
{
// wait until it is time to send this frame
std::this_thread::sleep_for(this_frame_time_slot - now - std::chrono::milliseconds(1));
}
++frames_since_sync_;
}
if ((rce_flags_ & RCE_FRAGMENT_PACING) && fps_)
{
if (!(rce_flags_ & RCE_FRAMERATE))
{
// if framerate is not use, we just use this moment as synchronization moment
this_frame_time_slot = std::chrono::high_resolution_clock::now();
}
std::chrono::nanoseconds packet_interval = frame_interval_/ active_->packets.size();
for (size_t i = 0; i < active_->packets.size(); ++i)
{
std::chrono::high_resolution_clock::time_point next_packet = this_moment + i * packet_interval;
std::chrono::high_resolution_clock::time_point next_packet = this_frame_time_slot + i * packet_interval;
// sleep until next packet time
std::this_thread::sleep_for(next_packet - std::chrono::high_resolution_clock::now());
@ -324,9 +341,10 @@ rtp_error_t uvgrtp::frame_queue::flush_queue()
return deinit_transaction();
}
inline std::chrono::high_resolution_clock::time_point uvgrtp::frame_queue::next_frame_time()
inline std::chrono::high_resolution_clock::time_point uvgrtp::frame_queue::this_frame_time()
{
return fps_sync_point_ + frame_interval_* (frames_since_sync_ + 1);
return fps_sync_point_ +
std::chrono::nanoseconds((uint64_t)(frames_since_sync_ * frame_interval_.count()));
}
void uvgrtp::frame_queue::update_rtp_header()

6
src/frame_queue.hh
View File

@ -147,7 +147,7 @@ namespace uvgrtp {
void set_fps(ssize_t enumerator, ssize_t denominator)
{
fps = enumerator > 0 && denominator > 0;
fps_ = enumerator > 0 && denominator > 0;
if (denominator > 0)
{
frame_interval_ = std::chrono::nanoseconds(uint64_t(1.0 / double(enumerator / denominator) * 1000*1000*1000));
@ -159,7 +159,7 @@ namespace uvgrtp {
void enqueue_finalize(uvgrtp::buf_vec& tmp);
inline std::chrono::high_resolution_clock::time_point next_frame_time();
inline std::chrono::high_resolution_clock::time_point this_frame_time();
transaction_t *active_;
@ -174,7 +174,7 @@ namespace uvgrtp {
int rce_flags_;
bool fps = false;
bool fps_ = false;
std::chrono::nanoseconds frame_interval_;
std::chrono::high_resolution_clock::time_point fps_sync_point_;

3
src/media_stream.cc
View File

@ -225,6 +225,9 @@ rtp_error_t uvgrtp::media_stream::create_media(rtp_format_t fmt)
media_ = nullptr;
return RTP_NOT_SUPPORTED;
}
// set default values for fps
media_->set_fps(fps_enumerator_, fps_denominator_);
}
rtp_error_t uvgrtp::media_stream::free_resources(rtp_error_t ret)

97
test/test_4_formats.cpp
View File

@ -135,7 +135,7 @@ TEST(FormatTests, h265)
cleanup_sess(ctx, sess);
}
TEST(FormatTests, h265_large_fps)
TEST(FormatTests, h265_fps)
{
std::cout << "Starting h265 test" << std::endl;
uvgrtp::context ctx;
@ -144,14 +144,103 @@ TEST(FormatTests, h265_large_fps)
uvgrtp::media_stream* sender = nullptr;
uvgrtp::media_stream* receiver = nullptr;
int framerate = 10;
if (sess)
{
sender = sess->create_stream(SEND_PORT, RECEIVE_PORT, RTP_FORMAT_H265, RCE_NO_FLAGS);
sender = sess->create_stream(SEND_PORT, RECEIVE_PORT, RTP_FORMAT_H265, RCE_FRAMERATE);
receiver = sess->create_stream(RECEIVE_PORT, SEND_PORT, RTP_FORMAT_H265, RCE_NO_FLAGS);
if (receiver)
{
sender->configure_ctx(RCC_FPS_ENUMERATOR, 100);
sender->configure_ctx(RCC_FPS_ENUMERATOR, framerate);
sender->configure_ctx(RCC_FPS_DENOMINATOR, 1);
}
}
std::vector<size_t> test_sizes = { 10000, 20000, 30000, 40000, 50000, 75000, 100000 };
// the default packet limit for RTP is 1458 where 12 bytes are dedicated to RTP header
int rtp_flags = RTP_NO_FLAGS;
int nal_type = 5;
rtp_format_t format = RTP_FORMAT_H265;
int test_runs = 10;
for (auto& size : test_sizes)
{
std::unique_ptr<uint8_t[]> intra_frame = create_test_packet(format, nal_type, true, size, rtp_flags);
test_packet_size(std::move(intra_frame), test_runs, size, sess, sender, receiver, rtp_flags, framerate);
}
cleanup_ms(sess, sender);
cleanup_ms(sess, receiver);
cleanup_sess(ctx, sess);
}
TEST(FormatTests, h265_small_fragment_pacing_fps)
{
std::cout << "Starting h265 test" << std::endl;
uvgrtp::context ctx;
uvgrtp::session* sess = ctx.create_session(LOCAL_ADDRESS);
uvgrtp::media_stream* sender = nullptr;
uvgrtp::media_stream* receiver = nullptr;
int framerate = 10;
if (sess)
{
sender = sess->create_stream(SEND_PORT, RECEIVE_PORT, RTP_FORMAT_H265, RCE_FRAMERATE | RCE_FRAGMENT_PACING);
receiver = sess->create_stream(RECEIVE_PORT, SEND_PORT, RTP_FORMAT_H265, RCE_NO_FLAGS);
if (receiver)
{
sender->configure_ctx(RCC_FPS_ENUMERATOR, framerate);
sender->configure_ctx(RCC_FPS_DENOMINATOR, 1);
receiver->configure_ctx(RCC_UDP_RCV_BUF_SIZE, 40 * 1000 * 1000);
receiver->configure_ctx(RCC_RING_BUFFER_SIZE, 40 * 1000 * 1000);
}
}
std::vector<size_t> test_sizes = { 1000, 2000, 3000, 4000, 5000, 7500, 10000 };
// the default packet limit for RTP is 1458 where 12 bytes are dedicated to RTP header
int rtp_flags = RTP_NO_FLAGS;
int nal_type = 5;
rtp_format_t format = RTP_FORMAT_H265;
int test_runs = 10;
for (auto& size : test_sizes)
{
std::unique_ptr<uint8_t[]> intra_frame = create_test_packet(format, nal_type, true, size, rtp_flags);
test_packet_size(std::move(intra_frame), test_runs, size, sess, sender, receiver, rtp_flags, framerate);
}
cleanup_ms(sess, sender);
cleanup_ms(sess, receiver);
cleanup_sess(ctx, sess);
}
TEST(FormatTests, h265_large_fragment_pacing)
{
std::cout << "Starting h265 test" << std::endl;
uvgrtp::context ctx;
uvgrtp::session* sess = ctx.create_session(LOCAL_ADDRESS);
uvgrtp::media_stream* sender = nullptr;
uvgrtp::media_stream* receiver = nullptr;
int framerate = 10;
if (sess)
{
sender = sess->create_stream(SEND_PORT, RECEIVE_PORT, RTP_FORMAT_H265, RCE_FRAGMENT_PACING);
receiver = sess->create_stream(RECEIVE_PORT, SEND_PORT, RTP_FORMAT_H265, RCE_NO_FLAGS);
if (receiver)
{
sender->configure_ctx(RCC_FPS_ENUMERATOR, framerate);
sender->configure_ctx(RCC_FPS_DENOMINATOR, 1);
receiver->configure_ctx(RCC_UDP_RCV_BUF_SIZE, 40 * 1000 * 1000);
@ -170,7 +259,7 @@ TEST(FormatTests, h265_large_fps)
for (auto& size : test_sizes)
{
std::unique_ptr<uint8_t[]> intra_frame = create_test_packet(format, nal_type, true, size, rtp_flags);
test_packet_size(std::move(intra_frame), test_runs, size, sess, sender, receiver, rtp_flags);
test_packet_size(std::move(intra_frame), test_runs, size, sess, sender, receiver, rtp_flags, framerate);
}
cleanup_ms(sess, sender);

20
test/test_common.hh
View File

@ -12,7 +12,7 @@ inline std::unique_ptr<uint8_t[]> create_test_packet(rtp_format_t format, uint8_
bool add_start_code, size_t size, int rtp_flags);
inline void test_packet_size(std::unique_ptr<uint8_t[]> test_packet, int packets, size_t size,
uvgrtp::session* sess, uvgrtp::media_stream* sender, uvgrtp::media_stream* receiver, int rtp_flags);
uvgrtp::session* sess, uvgrtp::media_stream* sender, uvgrtp::media_stream* receiver, int rtp_flags, int framerate = 25);
inline void send_packets(std::unique_ptr<uint8_t[]> test_packet, size_t size,
uvgrtp::session* sess, uvgrtp::media_stream* sender,
@ -93,9 +93,10 @@ inline void send_packets(std::unique_ptr<uint8_t[]> test_packet, size_t size,
EXPECT_NE(nullptr, sender);
if (sess && sender)
{
std::cout << "Sending " << packets << " test packets with size " << size << std::endl;
std::cout << "Sending " << packets << " test packets with size " << size
<< " and interval " << packet_interval_ms << "ms" << std::endl;
std::chrono::steady_clock::time_point start = std::chrono::steady_clock::now();
std::chrono::steady_clock::time_point start = std::chrono::high_resolution_clock::now();
for (unsigned int i = 0; i < packets; ++i)
{
if (i % 60 == 0 && send_app)
@ -149,11 +150,12 @@ inline void wait_until_next_frame(std::chrono::steady_clock::time_point& start,
// wait until it is time to send the next frame. Simulates a steady sending pace
// and included only for demostration purposes since you can use uvgRTP to send
// packets as fast as desired
auto time_since_start = std::chrono::steady_clock::now() - start;
auto next_frame_time = (frame_index + 1) * std::chrono::milliseconds(packet_interval_ms);
if (next_frame_time > time_since_start)
auto time_since_start = std::chrono::high_resolution_clock::now() - start;
auto next_frame_slot = (frame_index + 1) * std::chrono::milliseconds(packet_interval_ms);
if (next_frame_slot > time_since_start)
{
std::this_thread::sleep_for(next_frame_time - time_since_start);
std::this_thread::sleep_for(next_frame_slot - time_since_start);
}
}
@ -178,7 +180,7 @@ inline void cleanup_ms(uvgrtp::session* sess, uvgrtp::media_stream* ms)
}
inline void test_packet_size(std::unique_ptr<uint8_t[]> test_packet, int packets, size_t size,
uvgrtp::session* sess, uvgrtp::media_stream* sender, uvgrtp::media_stream* receiver, int rtp_flags)
uvgrtp::session* sess, uvgrtp::media_stream* sender, uvgrtp::media_stream* receiver, int rtp_flags, int framerate)
{
EXPECT_NE(nullptr, sess);
EXPECT_NE(nullptr, sender);
@ -188,7 +190,7 @@ inline void test_packet_size(std::unique_ptr<uint8_t[]> test_packet, int packets
{
Test_receiver* tester = new Test_receiver(packets);
int interval_ms = 10;
int interval_ms = 1000/framerate;
add_hook(tester, receiver, rtp_receive_hook);