For the past couple of weeks I've been working on the networking code for the CVE. Our network code transfers small packets well. However, I was unsure how the network code would react to large packets.

So, I set out to write a small test app to transfer large files. It only took a hour or so get a simple server and client working. Setting the packet size to around 8000 bytes didn't cause any problems. However, when I set the packet size to 16000 bytes, I noticed that while I would get the entire file, the md5 sum of the file would differ from the server's copy of the file.

Eventually, I discovered that the problem was that the code needed multiple calls to ::send to send the packet. I had already written code to account for the need to call ::send multiple times. Unfortunately, there was a small bug in it.

The file transfer protocol works something like this:

Client		Server
Request File	>
	<	File Size MD5 Sum

The following loop is then followed:

Client		Server
Last Received Chunk Number Request Size Request Location	>
	<	Chunk MD5 Sum Chunk Number Chunk Size Chunk Data

When the server receives a request for a chunk beyond the end of the file, it sends a message stating that the file is done. The client then makes sure that the MD5 sum of the downloaded file matches the sum of the file on the server. Another file can then be requested.

Using a packet size of 16000 bytes I was able to transfer files around 450k/s. This is a far cry from the 12MB/s that NFS transfers files on my home network.

Clearly I need to implement some kind of throttling mechanism. Guess I'm off to read some of Richard Steven's books.

Overall, I'm fairly happy with the CVE networking code. It ensures that data gets to the remote host in the order is was sent. The networking code also presents packets to the app level code as complete packets. This eliminates the worry of accidentally processing partial packets. Finally, the code is safe and easy to use.

For the CVE, transferring large files isn't a priority right now, but in the future, I believe it will be.