[crossfire] Protocol & compression.

[Sebastian Andersson]

On Sat, Mar 25, 2006 at 01:00:59AM -0800, Mark Wedel wrote:
>   For other reasons, I still think I'll add animation support to the map2 
> command.  However, I think it also worthwhile to add something like:
> 
> S->C: compress <data>

I've got some experience in this, having implemented it in a mud
(DUMII) and I suggest a different approach. Have the zlib stream as a
layer between the socket and the server/client when it is turned on
and start it like this:

C->S: start_compression
S->C: compression_started
The server now starts to compress everything sent to the client in a
layer below the output writing and the socket and after the client has
read "compression_started", it starts to decompress everything via zlib,
it becomes a layer below the normal send(2) on the server and below the
normal read(2) on the client.

To get things actually out of the zlib stream, one of course has to
call it with Z_SYNC_FLUSH. Doing that too often (like after every single
command) makes the compression rate much lower than it can be.

In DUMII we had already solved it when we added compression, since we
had previously added support for linux' TCP_CORK flag and later
the MSG_MORE flag (to avoid sending out tiny TCP packets). We did it
by adding a flag to each connection, let's call it NeedsFlush.
Every time something is sent to a client, the NeedsFlush flag is set.
Every time the expired timers or a player command has been processed,
we scan all connections to see which connections has that flag and
those that have are flushed (calling deflate with Z_SYNC_FLUSH and
the final send(2) is called without the MSG_MORE flag set) and
the NeedsFlush flag is reset.

For really tiny packets (the mud's 4 octet prompt for example), they
become somewhat larger, but that doesn't matter much since the TCP packet
header is so large and it is really rare that a single, short packet has
to be flushed (the player just pressing return without supplying a
command is one use-case, but that doesn't happen often), often many
tiny packets are compressed together and the resulting packet is much
smaller.

I guess this could be done with the crossfire server too, without too
much difficulty. The suggested compresion of selected packets and doing
them packet by packet, sounds like a bad solution to me since then
common combinations can't be compressed (and of course more overhead is
added by the "compress" prefix).

If the connection seems more laggy, one can add some counters, so a
flush is also performed after X compressed bytes, but I doubt it is
needed, since zlib can uncompress bytes even without comming to a
synchronisation point. It is probably better to use a smaller MTU in
those cases (either by using ioctls on the sockets on those OSes where
that is possible, or by simulating it by sending smaller packets and not
using Nagle's algorithm). But I doubt that will matter much and it will
not make things worse than they are without compression anyway.

Regards,
/Sebastian
-- 
                             .oooO o,o Oooo.  Ad: http://dum.acc.umu.se/
                             (   ) \_/ (   )      (o_
"Life is not fair, but root   \ (  /|\  ) / (o_   //\
password helps!" -- The BOFH   \_)     (_/  (/)_  V_/_