An easy way to find bugs

For work, I am building a video processing framework based on Akka actors. Among other things, I have to read and write packetised binary such as Mpeg2 Transport Stream or RTP packets.

Scala provides 2 techniques which makes this easier. I use case classes to model the various types of packet, and type classes to marshall / unmarshall the objects to and from a binary representation. Type classes can be used for many other things but serialisation (see this JSON example) is a very pragmatic use case.

Here is an example for an RTP Packet

/** Model an RTP Packet. See [[http://en.wikipedia.org/wiki/Real-time_Transport_Protocol]] for more details.*/
case class RtpPacket(
    marker : Boolean,
    payloadType : Byte,
    sequenceNumber : Short,
    timestamp : Int,
    ssrc :Int,
    csrcList : List[Int],
    extensionHeader : Option[RtpPacketExtensionHeader],
    payloadData : ByteString,
    paddingBytes : ByteString
){
  require(csrcList.length <= 15,s"An RTP packet can have between 0 to 15 CSRC elements. The provided list contains ${csrcList.length} elements.")
  require(paddingBytes.length <= 254 , "An RTP packet cannot have more than 254 bytes worth of padding.")
}

While the case class is straight forward, the corresponding type classes are a bit more complicated because they have to pack and unpack fields in bit, short, ints, longs, or other fields with arbitrary bit length value. This kind of code is difficult to write properly, a bit difficult to read and more importantly demands some serious testing.

Instead of generating test values by hand, I use a 3rd technique: property-based testing provided by the ScalaTest framework in combination with ScalaCheck.

Essentially, the testing framework generates test values automatically and asserts that what I serialize to binary must deserialize to the same value.

The code is as follows:

  "An arbitrary RtpPacket" must {
    "be correctly encoded / decoded to / from binary" in {
      
      val rtpPacketExtensionHeaderGen = for(
          option <- arbitrary[Option[Short]];
          bytes <- arbitrary[Array[Byte]].filter(a => a.length %4 == 0).map(a => ByteString(a))
      )yield option.map(id => RtpPacketExtensionHeader(id, bytes))
      
      val byteStringGen = arbitrary[Array[Byte]].map(a => ByteString(a))
      
      val rtpPacketGen = for(
     marker <- arbitrary[Boolean];
     payloadType <- Gen.choose[Byte](0,0x7F);
     sequenceNumber <- arbitrary[Short];
     timestamp <- arbitrary[Int];
     ssrc <- arbitrary[Int];
     csrcList <- Gen.listOfN(15,arbitrary[Int]);
     extensionHeader <- rtpPacketExtensionHeaderGen;
     payloadData <- byteStringGen;
     paddingBytes <- byteStringGen.filter( bs => bs.length <=254)
      ) yield RtpPacket(
          marker,
          payloadType,
          sequenceNumber,
          timestamp,
          ssrc,
          csrcList,
          extensionHeader,
          payloadData,
          paddingBytes
      )
      
      forAll(rtpPacketGen)({ packet =>
        val packed = ByteStringSerialization.write(packet)
        val parsed = ByteStringSerialization.read[RtpPacket](packed)
        parsed.paddingBytes must equal (packet.paddingBytes)
      })

There is not much code to write, yet the framework is ferociously efficient at finding bugs in my marshalling code. Here are some of the issues ScalaTest "found" for me:

• If I use a byte field to store the length of an array, surely the array must be less than 255 bytes
• A Scala byte is signed, so if you store a length value in a byte, don't forget to properly mask it back to an Int
• Stupid errors when I confuse bit offset with byte offset.
• etc...

Ever since I started using ScalaCheck, I've been amazed at its abilities to hone on bugs in my code. It is a truly powerful tool.