[crossfire] Protocol & compression.

[tchize]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just a note about the suggested
s->c compressed <data>
c->s compressed <data> (yeah imho shoud go in both directions)

if we assume data contains a compressed list of 1 or more commands, i
think the question on what need and what need not to be compressed os
not immediate and we can have various attempt with various server /
client version. The fact of assuming any command can be compressed by
algorithm X but that not all commands will be compressed, is enough to
write this protocol add-on. Then it's a matter of tuning the
compression triggers, but this can be done without breaking
interaction with previous versions. So imho, the 'when' to compress
should not be fixed in protocol, only the how is to be fixed.
One of the most important question is which algorithm do we use? You
said you gave a try with zlib, but which algorithm does it uses? Does
it involves a dictionnary? If yes, do we plan to reset the dictionnary
content between each compress command or do we plan to keep it from
begin to end?

My opinion currently is
 - assume client can receive any data in compressed mode, but that not
all datas are compressed (the sender has choice for every command)
 - assume the compression stream as something interleaved with not
compressed stream, identified by a specific marker (compressed)
 - assume the compressed stream continuous (same 'zlib/any chosen
algorithm' session, usefull considering number of repeated text messages)
 - assume setup negociate the algorithm (client say i support x,y,z
then server send ok, let's go for y algorithm, this is how http does it)

Regards

Mark Wedel a écrit :

> On the mailing list, irc, and elsewhere discussion came up about
> rather than trying to be too clever about minimize bandwidth with
> things like animations to just compress the data itself, as that
> keeps things simpler.
>
> Alex ran some quick tests, getting ~40% compression rate on big map
> payloads. However, that was client side, so I quickly hacked the
> server to do some compression on map packets (map1a). This way, I
> could also time how long it takes to do the compression (running on
> an athlon mp 2000 here).
>
> General notes - small map packets are not worth compressing. The
> cut off to be worth while is probably around 200 bytes.
>
> For a big map transfer, in this case, leaving the scorn apartment
> building with a 25x25 map in dm mode (so everything is visible),
> zlib compressed 3034 bytes to 1637 (47%) in 952 µsec. bz2 didn't
> do quite so good (1818) in 2113 µsec
>
> The other case I quickly tested was standing at the far north end
> of scorn (still in DM mode) to get a lot of ocean spaces. In this
> case, zlib compressed 806 bytes to 270 bytes (67%) in 505 µsec.
> bz2 compressed it to 245 in 408 µsec.
>
> Note that 1000 µsec = 1 ms, and by default, there is 120 ms/game
> tick. So spending a few milliseconds on compression doesn't seem
> to likely add much to the load (get enough players it adds up, but
> then so does the overall bandwidth compression).
>
> I'm personally think zlib is better in this case, as it is faster,
> and for big data, is smaller, and the interface is simpler.
>
> For other reasons, I still think I'll add animation support to the
> map2 command. However, I think it also worthwhile to add something
> like:
>
> S->C: compress <data>
>
> Where data is the block of compressed data. The length of data can
> be determined from the packet header (a shorter command, like comp
> or something could be done).
>
> <data> would get uncompressed on the client, and just contain
> normal protocol commands that the client then processes. By
> abstracting at a higher level, it allows us to compress any other
> protocol commands (think itemcmd for a large inventory. Or it
> would involve more work, but a long list of drawinfos, like from a
> shop listing).
>
> Thoughts/comments? On the server, this is pretty simple to do
> (simplest would be to add another parameter to send_with_handling
> on whether to compress the data) (some data, like png images,
> aren't worth trying to compress).
>
> For the client, it is more complicated, because you could have
> something like 'command; compressed data; command;' - right now, it
> justs uses a larger ring buffer IIRC. We don't want to have to
> move all the data beyond our compressed ata back - I suppose that
> since it is a ring buffer, we could backtrack the buffer to insert
> the uncompressed data. Or use a few buffers.
>
>
>
>
>
>
> _______________________________________________ crossfire mailing
> list crossfire at metalforge.org
> http://mailman.metalforge.org/mailman/listinfo/crossfire
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFEKB9QHHGOa1Q2wXwRAouFAKDKQdr4SKQFpEVDWltOGoBsoYzeFwCeO4Dn
tFKA2o4lip80LoXZQk6VZa8=
=fZWn
-----END PGP SIGNATURE-----