RTI Data Distribution Service

Latency

These graphs show the average one-way latency in microseconds for publish/subscribe messaging. Latency was measured by having the consumer (DDS DataReader) echo messages back to the producer (DDS DataWriter). This allowed roundtrip latency to be measured on the sending machine, avoiding time synchronization issues. The roundtrip latency was divided in half to get the one-way latency that is shown.

This data shows that, at small message sizes, latency remains consistently low. At larger messages sizes, which are network-limited, latency is proportional to message size.

Using higher-end hardware, RTI Data Distribution Service latency has been measured as low as 30 microseconds over Gigabit Ethernet.

Jitter

This graph is based on the same data as the previous latency graphs. It shows the same average latency (in red) but adds jitter—the variation in latency from message to message. A system is more deterministic if it exhibits lower jitter.

The two blue series show the minimum measured latency and the 99% latency (the latency below which 99% of the samples fell). Even at larger message sizes, the variation between minimum and 99% latency remains consistent at under 20 microseconds. This shows that RTI Data Distribution Service exhibits very low jitter and very high determinism, making it suitable for time- and mission-critical applications.

One-to-One Throughput - Bandwidth

This graph shows sustainable one-to-one (point-to-point) publish/subscribe throughput in terms of network bandwidth (megabits per second). It was measured between a single producing and consuming thread, each using a single Gigabit Ethernet port, and over a single DDS topic.

Accounting for Ethernet, IP and UDP overhead, the maximum bandwidth available for message data (and metadata) is slightly over 950 megabits. As can be seen, RTI Data Distribution Service is able to fully utilize all of this available bandwidth when sending messages larger than 128 bytes—meaning throughput is limited by the network and not by the CPU or RTI Data Distribution Service. This throughput is 25x higher than most other messaging and integration middleware.

Because RTI Data Distribution Service uses true peer-to-peer messaging—with no centralized or per-node Enterprise Service Bus (ESB), message broker, server or daemon processes—it does not impose any inherent limit on aggregate messaging capacity. It is limited only by the network infrastructure. So, in practice, RTI can deliver orders of magnitude higher capacity than other solutions. (See the topic and capacity scalability benchmark below for an illustration of this.)

One-to-One Message Rate

These graphs present the same throughput data in terms of the message rate (measured in messages per second), showing that RTI can support over 1,000,000 messages per second per-producer, per-consumer and per-topic.

One-to-Many Throughput Scalability

This graph shows the efficiency of RTI's reliable multicast protocol for one-to-many publish/subscribe messaging and real-time data distribution. A single producing thread was used to send a stream of 200-byte messages to up to 888 consumers, each running on a dedicated core with four consumers per NIC. Going from one to 888 subscribers had less than a 13% impact on maximum throughput for the topic.

This demonstrates that RTI Data Distribution Service is highly scalable in terms of the number of consumers that can be supported on a given topic. The number of consumers has no significant impact on a topic's sustainable throughput. (RTI also provides flexible and automatic handling of slow consumers for cases in which some subscribers cannot keep up with the message rate.)

Note that this benchmark was conducted on different hardware than the others, using Intel Xeon processors.

Impact of Throughput on Latency

Throughput's impact on latency is an essential measure of a messaging solution's suitability for time-critical applications. For many applications, latency is most critical when message rates (and throughput) increase.

• For an automated trading application, increases in market data rates are often a sign of volatility. If a trading infrastructure loses messages or cannot keep up during bursts, decisions could be made too late or based on inaccurate or stale information. If lost messages are not well handled, an entire system could be brought down, taking you out of the market when it is most critical to be in.
• In a combat system, when the number of radar tracks increases it can be a sign of a significant threat. The data distribution infrastructure must be able to accommodate worst-case volumes while still meeting the real-time deadlines necessary for an accurate and in-time response.

Enterprise messaging middleware typically queues messages (or blocks producers) when volume exceeds capacity. In contrast, RTI Data Distribution Service is designed for real-time applications in which the consequences of excessive latency could be catastrophic.

This graph shows RTI's sustainable average latency as a function of message rate for 200-byte messages. Average latency remains low, staying under 250 microseconds, even near network saturation. With most other messaging middleware, latency jumps to several milliseconds or more during bursts—if they can be handled at all.

Impact of Throughput on Jitter

This graph is based on the same data as the previous, with measures of jitter added. The dashed red lines indicate standard deviation and the blue line show the latency below which 99% of the samples fell. It illustrates that RTI's real-time performance is still well-bounded even under extreme load.

Impact of Throughput on One-to-Many Latency and Scalability

This graph shows that, when there are multiple subscribers to the same topic, RTI's reliable multicast protocol maintains low latency in addition to high throughput (as was illustrated in the one-to-many scalability benchmark). The red series show the impact of latency on throughput with one consumer subscribed to the topic, the green with 20 and the blue with 40. Latency at a given throughput varies very little as the number of consumers increases.

What variation there is between 20 and 40 subscribers can mainly be attributed to NIC and operating system contention, since there were two subscribers per NIC in the 40-subscriber benchmark but one per NIC in the others.

Topic and Capacity Scalability

This benchmark illustrates the effect that the number of topics has on latency, throughput and system capacity.

The green series shows the relationship between throughput and latency when there is a single topic with one producer and one consumer (this is the same data as in the Impact of Throughput on Latency graph). The red series shows the relationship between throughput and latency when there are eight topics, each with a single producer and consumer. Messages per second is the aggregate across all eight topics. The load was equally balanced across all eight topics and latency is averaged over all eight consumers.

These results show that capacity scales proportionally with the number of topics—without impacting latency. Peak throughput with one topic was 563,937 MPS and with eight topics it was almost exactly eight-times this: 4,505,730 MPS. In both cases, average latency was about 275 microseconds.

This illustrates that:

• systems can be partitioned across multiple topics to achieve extraordinarily high aggregate throughput while maintaining very low latency and
• new topics can be added to a system without impacting latency or throughput on other topics (attributable to RTI's peer-to-peer architecture with no centralized brokers or ESBs as bottlenecks).

To simplify system design and achieve a similar effect, RTI also allows a single topic to be partitioned across multiple multicast groups. This allows per-topic throughput to exceed the capacity of a consumer or network port. Consumers that do not want all messages or that cannot sustain the topic's aggregate throughput can use a filter to specify the messages they want—corresponding to a DDS ContentFilteredTopic or JMS Message Selector. The filter determines the multicast addresses that are subscribed. IGMP snooping is supported so that undesired messages can be filtered by the switch.