[crossfire] Protocol & compression.

[Mark Wedel]

Alex Schultz wrote:
> Mark Wedel wrote:
> 
>>  General notes - small map packets are not worth compressing.  The cut off to 
>> be worth while is probably around 200 bytes.
>>  
>>
> One note, usually, map packets are larger than that unless they are 
> animation ones.

  Depends - I see lots of small packets.  If your just standing around in one 
place, the only thing being sent are the animations, which can be pretty small.

  I'd also imagine that if you are moving around on a dark map with only a few 
spaces visible, you'll get pretty small packets.


> Perhaps system load could be measured in determining what threshold of 
> what packet size should be before it is compressed, with a hard minimum 
> size set as well (so, if the system load gets high, only compress the 
> REALLY big packets, which should be rarer). This isn't the simplest to 
> implement, but perhaps if reading the system load directly doesn't work 
> well, perhaps measuring how close the ticks are to actually hitting 120 
> ms/game tick, and if they start taking too long, then scale the 
> compression to back to be used on fewer packets.
> Neither of these methods is the easiest ways to do it, but IMHO it would 
> be the 'smartest' way to do it.

  measuring ms would be the way to go.  System load is meaningless on multi 
processor systems (and with dual core cpus, those are becoming more an more 
common).  does a 1.0 load average mean that the crossfire server is using all 
the cpu time, or does it mean some other program is just caught in an infinite 
loop and crossfire in fact isn't spending much time at all.

  However, if we were going to go down the idea of adapting what the server does 
base on available time, there are lots of other things.  Don't swap out maps if 
busy.  Put off for a few ticks the routine stuff (in do_specials()).  But that 
also starts getting more complicated - some players may not care much about 
compression but don't mind using it to help reduce server bandwidth, etc.  Yet 
others on low bandwidth connections really want that compression - this could 
lead to things like clients saying how badly they want compression.  Not sure if 
that complication is worth it.

  The other problem I think with that approach is that the bigger the packet, 
the longer it takes to compress.

  but the biggest issue is this scenario - player changes maps - loading this 
new map takes time, so server is now short on time.  Yet because player is 
changing maps, a lot of map data is changing, so that is the packet you want to 
compress.


>> S->C: compress <data>
>>
>>  Where data is the block of compressed data.  The length of data can be 
>> determined from the packet header (a shorter command, like comp or something 
>> could be done).
>>
>>  <data> would get uncompressed on the client, and just contain normal protocol 
>> commands that the client then processes.  By abstracting at a higher level, it 
>> allows us to compress any other protocol commands (think itemcmd for a large 
>> inventory.  Or it would involve more work, but a long list of drawinfos, like 
>>from a shop listing).
>>  
>>
> This makes sense to me. Perhaps the server should compress automatically 
> for ALL packet types where the packet is big enough unless:
> 1) Otherwise flagged (png data)

  Well, my thinking was the send_with_handling() will be passed a flag on 
whether to compress or not.  Note that compression also has to be negotiated by 
the client, so the flag would really be 'if this is set, this is compressible 
data'.  send_with_handling() would still see if it meets minimum size, if the 
client supports compression, etc.

> 2) Server load is becoming too high (perhaps measure by measuring the 
> actual number of ms between ticks).

  Maybe, but as mentioned above, if we're going to do that, there is probably 
lots of code to look at within the server for 'should we do this, should we do 
that' type of logic which doesn't currently exist.