Home > technical > The Angle Bracket Tax

The Angle Bracket Tax

The following C++14 code fragment represents a general message layout along with a specific instantiation of that message:

Message Code Frag

 

Side note: Why won’t a C++98/03 compiler accept the above code?

Assume that we are “required” to send thousands of these X-Y position messages per second between two computers over a finite bandwidth communication link:

Sensor To Display

There are many ways we can convert the representation of the message in memory into a serial stream of bytes for transmittal over the communication link, but let’s compare a simple binary representation against an XML equivalent:

XML Vs Binary

The tradeoff is simple: human readability for performance. Even though the XML version is self-describing and readable to a human being, it is 6.5 times larger than the tight, fixed-size, binary format. In addition, the source code required to serialize/deserialize (i.e. marshal/unmarshal) the XML version is computationally denser than the code to implement the same functionality for the fixed-size, binary representation. In the software industry, this tradeoff is affectionately known as “the angle bracket tax” that must be payed for using XML in the critical paths of your system.

If your system requires high rates of throughput and low end-to-end latency for streaming data over a network, you may have no choice but to use a binary format to send/receive messages. After all, what good is it to have human readable messages if the system doesn’t work due to overflowing queues and lost messages?

  1. September 8, 2014 at 4:47 am

    Side note:

    Uniform initialisation was introduced to C++11. It’s very lovely, but can be confusing when mixed with old code. Prior to C++11, the correct invocation would be

    Message current_position = {4, 56101, 723674};

    Also, the int32_t type was standardised only then, although available on several platforms as a vendor specific implementation.

  2. September 8, 2014 at 2:46 pm

    How about using (binary) serialization, like one of these:
    https://kentonv.github.io/capnproto/news/2014-06-17-capnproto-flatbuffers-sbe.html

    Human readability is another matter, but should help with performance 😉

    • September 8, 2014 at 3:15 pm

      Two words: OMG DDS 🙂

      • September 8, 2014 at 3:43 pm

        Googled, found OMG’s Data Distribution Service (DDS), looks interesting, thanks! 🙂 Out of curiosity, would you happen to know how does it fare in comparison with the aforementioned serialization formats?

  3. Gerald
    September 8, 2014 at 9:46 pm

    Two more words: msgpack python

  4. September 9, 2014 at 5:33 am

    mttpd, I think it’s like comparing apples to oranges. DDS is an industrial strength, full blown, distributed communication subsystem that uses binary serialization/deserialization (CDR format) while protobufs, Boost.Serialize, etc, are simply binary serialization/deserialization implementations. DDS is standardized and has many vendors who supply implementations that are interoperable with each other.

    There are open source versions of DDS that you can play with: OpenDDS and OpenSplice. Also check out my “hands on” article about my experience playing with RTI’s stellar implementation of DDS: https://bulldozer00.com/dds-hands-on/

    Cheers!

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: