Difference between revisions of "DTLS Implementation Notes"

Revision as of 23:26, 8 January 2009

This page documents the experience of implementing SNMP over DTLS as described by documents being developed for the ISMS working group. A large section of this is relevant only to SNMP developers (Net-SNMP or otherwise) and some of this is relevant to anyone who is implementing a DTLS solution using OpenSSL. In particular, there are a number of tricks that need to be employed to make OpenSSL properly handle multiple clients. The implementation and this document were done by Wes Hardaker

Net-SNMP Background

Net-SNMP implements the transports over which SNMP messages can be sent using a pluggable architecture. This architecture defines hooks that allow implemented transports to handle opening, sending and receiving packets through "something or other".

To implement DTLS support within Net-SNMP a new transport had to be written that was responsible for sending and receiving packets. This new file (snmplib/snmpDTLSUDPDomain.c), though not yet checked into the SVN repository as of the time of this writing, is functional and has passed a number of tests. It should hopefully be ready for release by the Net-SNMP 5.5 release.

OpenSSL Background

OpenSSL's internal implementation architecture is well designed from a modular point of view. To some extent, however, this will come back to bite us as we'll see later on.

TLS/DTLS

Internally the TLS and DTLS implementations merely process the data they receive through "anything" and send responses back through the configured mechanism. These sending and receiving mechanisms are entirely separated from the TLS/DTLS implementations by a modular data sending and receiving framework called BIOs.

BIOs

BIOs in OpenSSL are merely code that knows how to send and receive data. You could easily think of them as a buffering layer between a data producer and consumer and where-ever-the-data-needs-to-go. They act in a very similar fashion to the way transports separate the Net-SNMP packet processing from the sending/receiving framework.

BIOs exist in OpenSSL to handle sending and receiving from many different directions. Probably the most common BIO is the one that attaches to a network socket like the TCP BIO. There are also BIOs that wrap around stdout, and there are memory BIOs that merely store data in a buffer. Starting with some version of OpenSSL there also was a datagram BIO that was designed to be used with DTLS and UDP. The datagram BIO is merely a wrapper around UDP sockets in the same way that the TCP BIO wrapped around TCP sockets.

TCP vs UDP Background

There is a fundamental difference, of course, between how UDP and TCP works. The biggest difference is in receiving packets. Normally for TCP you have to call accept() to allow a new connection to come through and get a new OS socket for sending and receiving on that socket to that peer. You have no idea yet how important that last statement is, so let me repeat it. In bold. TCP implementations provide consumers with one socket per connection.

Within the OpenSSL framework, when a new connection is requested from a new client the OpenSSL stack also creates a new BIO which is then attached to the new socket.

With UDP there is only one socket available to send and receive from (well, you could create multiple sockets with a different socket per client using a different UDP port per socket but you'd have to convince all your clients to send the traffic to the newly opened port just for them... In fact this is sort of what FTP does). With UDP a receiving server needs to check each packet that it's getting data from for the source address. Then if it needs to respond to the packet it needs to make sure that it remembers the address so it can send the response back to the right place. (TCP implementations, on the other hand remember it for you so there is less to do).

OpenSSL Context Background

OpenSSL needs to keep state with respect to ever TLS or DTLS session that it has established. It does this through the use of a SSL * pointer which is then attached to the sending and receiving BIO * pointer using a call as follows:

         SSL_set_bio(SSL *ptr, BIO *read_from_bio, BIO *write_to_bio);

Then, whenever stuff needs to be done (be it sending data or negotiating cryptography) the internal TLS implementation, when operating on traffic or data being sent in the context of the ptr function can simply send and receive traffic destined for it using the two "tied" BIOs. Normally, as I mentioned earlier, these BIOs are wrappers around TCP sockets for use with TLS. But wait, there's more...

Where the problem comes in

Problem 1: UDP is a single socket

The first problem is that UDP uses only a single socket for sending and receiving. OpenSSL has to send and receive from multiple locations. IE, it needs multiple SSL * / BIO * pairs. I thought, originally, that maybe I could trick openssl and manually tell it which SSL * pointer to use for each packet. IE, I would do a PEEK at the incoming packet and based on where it was coming from (the remote source address and source port) I could pick the right SSL * pointer and tell OpenSSL to process the incoming packet. That way I could have just one DGRAM based BIO (like all the OpenSSL examples use) and still handle multiple connections.

This actually almost worked, except for the next problem got in the way:

Problem 2: Packets Pile Up

When OpenSSL is operating, particularly when starting up a new TLS or DTLS session, it needs to read and write multiple packets in order to complete negotiations. Internally OpenSSL will attempt to read as much as it can from the BIO it was given in order to process as much as possible before returning control to the application (possibly with data).

In the case of TLS over TCP, this is perfectly safe since the the BIO is attached to a socket where all the data coming through the socket should be destined for that one TLS connection (and if it's not, it's safe to throw away).

However! This concept of reading as much as you want fails completely when using UDP. The first packet may certainly be destined for the current SSL * context (especially if the application handed it carefully to OpenSSL as described above). But there is no guarantee that the next packet will be from the same client or the same TLS session.

So, when OpenSSL is trying to be as proactive as it can and read as much as it can from a socket before returning data to the calling application, it may actually read multiple packets from the BIO attached to the UDP socket and assumes they're all destined for the SSL * context that it's currently handling. This is a completely invalid thing to do, unfortunately. And in fact when I launched two DTLS instrumented versions of snmpget against a single DTLS instrumented snmpd server one of the clients failed because it was suddenly trying to read too much data from the UDP socket and read packets for one SSL * context that were, in fact, destined for the other.

The Solution That Works

Eventual Solution and Architecture Changes for OpenSSL

??