I've been playing with the new I/O features in Akka 2.2 which are really neat when it comes to receiving packets (presented as Akka ByteString) over UDP.
The RTP protocol, although not very complicated, requires a fair amount of bit-level manipulation to extract useful information from the UDP packets. If you are familiar with bit-level operations, it is not difficult to write code to parse packets, however, that code is a bit difficult to read. When you get into RTCP or H264 byte streams, things get a lot more complicated, so I am looking for ways to keep the code easy to process.
I've been experimenting with a little utility class for bit-level manipulation, based on Akka ByteStrings.
Here is what it looks like:
The stuff works quite well, but compared to a hand-written parser, it is a bit slower (just under twice as slow). I have not done Scala Macros before, but I have the feeling that this is the kind of computation which could be moved to compile-time in a macro. Basically turning the bit-shifting code into a set of instructions which can be optimised at compile time. Ideally, something like Erlang's bit pattern matching, although I don't know if it is possible in Scala.
I will try that next.
Cool man!
ReplyDeleteLong time ago I had to develop a Java server listening to C/C++ clients using a binary "struct based" protocol. For that I used a library called Javolution, in particular the Struct class that allowed me to program Java classes just as if they were the equivalent C/C++ struct:
http://javolution.org/
http://javolution.org/core-java/target/apidocs/javolution/io/Struct.html
In my particular case I did not have to support bit fields, but they are easily supported with a solution much like yours, for instance:
import java.nio.ByteBuffer;
class Clock extends Struct { // Hardware clock mapped to memory.
Unsigned16 seconds = new Unsigned16(5); // unsigned short seconds:5
Unsigned16 minutes = new Unsigned16(5); // unsigned short minutes:5
Unsigned16 hours = new Unsigned16(4); // unsigned short hours:4
Clock() {
setByteBuffer(Clock.nativeBuffer(), 0);
}
private static native ByteBuffer nativeBuffer();
}
Means that seconds take 5bits, the next five is minutes and hours take 4bytes.
The good thing about this was that I could reason about the messages just as the C/C++ team did on the client side. It even allowed me to forget about byte position cause the order of definition on the class determined the order of memory allocation in the "struct".
But not all of this was so cool. Apart from being just boring-old Java, the use of ByteBuffers, and its tidings with NIO, to support all of this complicated the read/write operations in my opinion needlessly. I guess you don't have that problem with scala+akka and ByteStrings.
Thanks for sharing!
Hola,
ReplyDeleteThanks for the information about Javolution. I was not aware of it. I've eventually worked out a solution using Scala Macros. It took me a while to grasp the syntax, but I think I now get it. I've been able to convert the method boolean(offset : Int) into a set of instructions that are ran are compile time. The code actually runs faster which is what I was hoping for. I shall blog about it (and maybe set up a proper github project) when I've made more progress.