[crossfire] Protocol & compression.

[Mark Wedel]

  On the mailing list, irc, and elsewhere discussion came up about rather than 
trying to be too clever about minimize bandwidth with things like animations to 
just compress the data itself, as that keeps things simpler.

  Alex ran some quick tests, getting ~40% compression rate on big map payloads. 
  However, that was client side, so I quickly hacked the server to do some 
compression on map packets (map1a).  This way, I could also time how long it 
takes to do the compression (running on an athlon mp 2000 here).

  General notes - small map packets are not worth compressing.  The cut off to 
be worth while is probably around 200 bytes.

  For a big map transfer, in this case, leaving the scorn apartment building 
with a 25x25 map in dm mode (so everything is visible), zlib compressed 3034 
bytes to 1637 (47%) in 952 µsec.  bz2 didn't do quite so good (1818) in 2113 µsec

  The other case I quickly tested was standing at the far north end of scorn 
(still in DM mode) to get a lot of ocean spaces.  In this case, zlib compressed 
806 bytes to 270 bytes (67%) in 505 µsec.  bz2 compressed it to 245 in 408 µsec.

  Note that 1000 µsec = 1 ms, and by default, there is 120 ms/game tick.  So 
spending a few milliseconds on compression doesn't seem to likely add much to 
the load (get enough players it adds up, but then so does the overall bandwidth 
compression).

  I'm personally think zlib is better in this case, as it is faster, and for big 
data, is smaller, and the interface is simpler.

  For other reasons, I still think I'll add animation support to the map2 
command.  However, I think it also worthwhile to add something like:

S->C: compress <data>

  Where data is the block of compressed data.  The length of data can be 
determined from the packet header (a shorter command, like comp or something 
could be done).

  <data> would get uncompressed on the client, and just contain normal protocol 
commands that the client then processes.  By abstracting at a higher level, it 
allows us to compress any other protocol commands (think itemcmd for a large 
inventory.  Or it would involve more work, but a long list of drawinfos, like 
from a shop listing).

  Thoughts/comments?  On the server, this is pretty simple to do (simplest would 
be to add another parameter to send_with_handling on whether to compress the 
data) (some data, like png images, aren't worth trying to compress).

  For the client, it is more complicated, because you could have something like 
'command; compressed data; command;' - right now, it justs uses a larger ring 
buffer IIRC.  We don't want to have to move all the data beyond our compressed 
ata back - I suppose that since it is a ring buffer, we could backtrack the 
buffer to insert the uncompressed data.  Or use a few buffers.