[crossfire] Protocol & compression.

[Mark Wedel]

Sebastian Andersson wrote:
> On Tue, Mar 28, 2006 at 10:57:21PM -0800, Mark Wedel wrote:
>> 3) Compress everything sent.  This should be done at a lower level (when we 
>> actually write to the socket).
>> Cons: Stille harder to do - if we compress a block of data, but can only write 
>> half to the socket, we have to put the other have in a 'this data is compressed 
>> and send it next' buffer (current logic just moves the pointer in the ring 
>> buffer).  We may end up compressing more data than we need - isn't really a 
>> convenient way to turn on/off compression.
> 
> I've only glanced at the code, but is it not simply a matter of renaming
> Write_To_Socket to Really_Write_To_Socket, and then create a new
> Write_To_Socket that calls Really_Write_To_Socket if compression isn't
> turned on for the client? If it is turned on, put the output data in the
> z_stream (z_stream->avail_in = len, next_in = (Bytef*)buf), loop over
> deflate until avail_in == 0, and anything in the compression buffer is
> sent to Really_Write_To_Socket, thus sending or storing it like usual.
> There would also have to be a flush function to call deflate with
> the Z_SYNC_FLUSH flag to flush after the tick has been processed.
> 
> To me, that seems like the simplest thing since the least amount of code
> would have to be changed and without compression turned on, the affected
> code paths are trivial to verify. The socket_struct would have to get
> a new z_stream* field and an extra buffer to store compressed data in
> (1024 bytes perhaps?).

  It isn't really hard to compress everything, but still harder than the first 
case I mentioned.  The first case, add parameter to Send_With_Handling about 
compression, is probably about 5-10 lines of code, and that is the only code 
needed.  No changing of structures.  No renaming of functions.  No extra 
buffers.  No need to add calls to flush buffers, etc.

  However, it seems likely that a compress everything approach will be done at 
some point.  But it certainly isn't the simplest.

> And while looking at the networking code...
> Perhaps I've missed something, but isn't Nagle's algorithm disabled
> (TCP_NODELAY)?  That makes the final message of each tick lag for up
> to 200ms on at least linux servers. On the other hand, with Nagle's
> algorithm disabled the server will probably send a lot of short packets.
> A hack, that might work is to turn off TCP_NODELAY, process a tick and
> then turn on TCP_NODELAY. At least linux servers will send any queued
> data when TCP_NODELAY is turned on. If the server supports TCP_CORK
> (linux-2.2 and newer, I think), that should be used instead, turn it on
> before the tick is processed and turn it off afterwards.
> (FreeBSD 4.5 and has a semanticly equivalent TCP_NOPUSH option, other
> BSD versions has, as far as I know, an incompatible TCP_NOPUSH that do
> not send the queued data until there is something more to send).

  At some level, a strong case for redoing most of the socket code could be made.

  However, you are correct that TCP_NODELAY really should be used.  But you are 
also right in that if we just enabled, we'd send a lot of small packets.

  I'm not 100% sure, but I'd think if we did something like:

setsockopt(.., TCP_NDELAY, 1)
write_to_socket("hello")
setsockopt(..., TCP_NDELAY, 0)

  (hello should be replaced by something else), but I'd think that would cause 
all data currently in the TCP buffer to get flushed out.  Since there has also 
been a discussion that the server should send info for each tick it processes 
(so animations and other stuff can remain in sync), sending that instead of the 
hello would probably work fine, and kill two birds with one stone, so to speak.

  I'm not particularly fond of using features specific to one OS - starts to 
make the code messier.  And generally speaking, I don't think TCP_CORK should 
really be necessary - basically, the code figures out everything to send to one 
player at once, sends it, then moves to the next player, etc.  So all that data 
should be generated at a pretty small time gap.

  the only exceptions I can think of this are shout/tell/says messages (as one 
player instigates them it sends them to the other players at that time), and 
sound events (sort of same reason, but map2 should move the sound events into 
that command).