[CF-Devel] Compression

crossfire-devel at archives.real-time.com crossfire-devel at archives.real-time.com
Fri Oct 15 13:03:44 CDT 2004


I suggest against UDP based protocal use. 1 it's
unreliable. 2 I wouldn't beable to connect through
ssh.
Please not UDP, I'm not _That_ worried about bandwidth
consumption. I am VERY glad CF uses TCP.

--- tchize <
     
     tchize at myrealbox.com
     
     > wrote:

>
     
      Blind compression of protocol is a bad idea. 
     
     >
     
      Here is why.
     
     >
     
      There are mainly 3 parts of protocol which consume
     
     >
     
      lots of bandwidth and, so 
     
     >
     
      are interresting to compress.
     
     >
     
     
     >
     
      1) text messages to player.
     
     >
     
      When use hit/miss a monster, server send lots of
     
     >
     
      message telling player how 
     
     >
     
      combat is going. Those message are all in forms like
     
     >
     
      'You hit kobold very hard'
     
     >
     
      Don't forget that is one packet as a whole, so you
     
     >
     
      need to find a a way to 
     
     >
     
      compress that sentence alone. Try to cut an past it
     
     >
     
      in a .txt then zip it and 
     
     >
     
      see result (yeah, compressed size is the double,
     
     >
     
      because you need to send the 
     
     >
     
      compression alphabet used). A bettr way would be,
     
     >
     
      perhaps to send only to 
     
     >
     
      client the number of the creature being hit (an
     
     >
     
      int16) along with the 
     
     >
     
      'template' used for message (could be another
     
     >
     
      int16). Never investigated a 
     
     >
     
      lot this.
     
     >
     
      2) pictures.
     
     >
     
      Depending on configuration of client, the client may
     
     >
     
      cache pictures or not, 
     
     >
     
      even could comme with a standard preloaded set of
     
     >
     
      pictures or could download 
     
     >
     
      the pictures from server at startup. As pictures are
     
     >
     
      already png, unless you 
     
     >
     
      force client to cache pictures (this prevent sending
     
     >
     
      pictures each time a 
     
     >
     
      square change in map), you can't gain a lot in this
     
     >
     
      way. (Preloaded set with 
     
     >
     
      'on connect' checksum / download seem the best path)
     
     >
     
      3) Mapinfo
     
     >
     
      There are lots of map related info send to client,
     
     >
     
      when he moves, when he is 
     
     >
     
      looking at the animated sea (each animation is
     
     >
     
      change in the map and so a map 
     
     >
     
      packet sent). I already analyzed this in the past,
     
     >
     
      and i concluded, by 
     
     >
     
      running with a quite fast player, in bigworld
     
     >
     
      (pictures were cached at 
     
     >
     
      connect time and so didn't influence bandwidth use,
     
     >
     
      statswere at max and 
     
     >
     
      didn't influence bandwidth too, no monsters around),
     
     >
     
      that the bandwidth use 
     
     >
     
      can sometime rise up to 32K/s just for map datas.
     
     >
     
      Unfortunately, as is, it's also impossible to
     
     >
     
      compress that data. These are 
     
     >
     
      already tight binary datas and, when you have
     
     >
     
      recurrent informations, like 
     
     >
     
      when you have a cyclic sea movement, you can't
     
     >
     
      gather all them to 'hope' zlib 
     
     >
     
      will compress them. Because you'll have to stack
     
     >
     
      datas for about 2 seconds 
     
     >
     
      (imagine the lags).
     
     >
     
     
     >
     
     
     >
     
      Now, am not saying we can't compress datas, just
     
     >
     
      that we need a better way, 
     
     >
     
      which has already been discussed for about a year on
     
     >
     
      irc. The idea is mainly 
     
     >
     
      around the mapdatas , but not only. The purpose is
     
     >
     
      not to reduce bandwidth 
     
     >
     
      (but it indeed reduce bandwidth) but to prevent user
     
     >
     
      lags. That's what we 
     
     >
     
      need to concentrate on. The suggestion lead to the
     
     >
     
      following point
     
     >
     
     
     >
     
      1) Use an udp based protocol. This prevent the
     
     >
     
      bandwidth consumption of tcp 
     
     >
     
      packet resend query, tcp stacking and so on, but
     
     >
     
      this protocol is not 
     
     >
     
      reliable, so we need to add redundancy on critical
     
     >
     
      datas (we don't mind 
     
     >
     
      loosing a 'you hit kobold' message or a 'sea did
     
     >
     
      animate' one), but when 
     
     >
     
      player moves, we need to ensure client got the
     
     >
     
      information. When player 
     
     >
     
      changes map we need to be sure it get this
     
     >
     
      information too. When players are 
     
     >
     
      chatting also, this must be reliable.
     
     >
     
     
     >
     
      2) Because there can be lag between a 'move' command
     
     >
     
      from client and a 
     
     >
     
      'mapscroll' answer from server, client need to take
     
     >
     
      some advance in the 
     
     >
     
      movement. It need to 'suppose' the movement, thought
     
     >
     
      the server remains the 
     
     >
     
      only controller of the 'can move' logic. This mean
     
     >
     
      we must send additionnal 
     
     >
     
      datas on which areas are passable and which not. (or
     
     >
     
      at least 'seems 
     
     >
     
      passable').
     
     >
     
     
     >
     
      3) Because animations on maps are a waste of
     
     >
     
      bandwith (cyclic animation, like 
     
     >
     
      sea, rivers, lava, snow), we need to tell the client
     
     >
     
      the animation exists, 
     
     >
     
      the speed of the animation and a timebase to
     
     >
     
      synchronize it's first frame. 
     
     >
     
      That's a bit more datas but sent only once.
     
     >
     
     
     >
     
      4) Because sending additionnal datas for an
     
     >
     
      animation player will merely see 
     
     >
     
      only half because he just run by when goind
     
     >
     
      somewhere else and because most 
     
     >
     
      of the envirronement never change (trees remain
     
     >
     
      trees, grass remains grass, 
     
     >
     
      lava, etc) we should preload to client the maps
     
     >
     
      backgrounds.
     
     >
     
     
     >
     
      5) Because we need to keep all dynamism of current
     
     >
     
      map system (a grass square 
     
     >
     
      could theorically be changed by weather system. You
     
     >
     
      could create a map with 
     
     >
     
      scripting where a lever would remove lava, aso) we
     
     >
     
      need to have a way to 
     
     >
     
      'patch' the client map when such change occur and a
     
     >
     
      way to track this.
     
     >
     
     
     >
     
      All this need to make a list of important datas, of
     
     >
     
      less likely to change info 
     
     >
     
      on map (should be automatic and based on
     
     >
     
      is_floor/alive/can_pick tags).
     
     >
     
      Moreover, this need a redo of current protocol which
     
     >
     
      would not be compatible 
     
     >
     
      (well technically because we would use udp and not
     
     >
     
      tcp, we can easily decide 
     
     >
     
      if we use new or old protocol with client).
     
     >
     
     
     >
     
      So i think this is not as simple as compression,
     
     >
     
      this is some quite huge work 
     
     >
     
      that need to be done and turn around in my mind for
     
     >
     
      more than a year. But i 
     
     >
     
      sure hope to see this alive one day or another :)
     
     >
     
     
     >
     
     
     >
     
     
     >
     
     
     >
     
      Le Lundi 11 Octobre 2004 20:05, Ben Jolitz a écrit :
     
     >
     
      I think that is an excellent idea. Perhaps using the
     
     >
     
      zlib compression,
     
     >
     
      which is standard, you can send more data as
     
     >
     
      quickly.
     
     >
     
      I frequently experiance lag on broadband. if Cf
     
     >
     
      worked on dial up,
     
     >
     
      think how much better. Perhaps using a system
     
     >
     
      library,
     
     >
     
      like libz, is better. But having compressed maps
     
     >
     
      sent over is really neat.
     
     >
     
      Ben
     
     >
     
     
     >
     
      >I was wondering if on-the-fly gzip compression
     
     >
     
      could
     
     >
     
      >be added to crossfire, since much of the traffic is
     
     >
     
      >text based map data. Routienes could be pulled from
     
     >
     
      >mozilla or midnight commander to implement it.
     
     >
     
      >
     
     >
     
      >
     
     >
     
      >
     
     >
     
      >Thoughts? (Would speed things up for dialup users).
     
     >
     
     
     >
     
      _______________________________________________
     
     >
     
      crossfire-devel mailing list
     
     >
     
     
      crossfire-devel at lists.real-time.com
      
      
     >
     
     
     
     https://mailman.real-time.com/mailman/listinfo/crossfire-devel
     
     
>
     
     
     >
     
      -- 
     
     >
     
      --
     
     >
     
      Tchize (David Delbecq)
     
     >
     
     
      tchize at myrealbox.com
      
      
     >
     
      Public PGP KEY FINGERPRINT:
     
     >
     
          F4BC EF69 54CC F2B5 4621  8DAF 1C71 8E6B 5436
     
     >
     
      C17C
     
     >
     
      Public PGP KEY location:
     
     >
     
     
      http://wwwkeys.pgp.net/pgpnet/wwwkeys.html
      
      
     >
     
     
     
>
     
      ATTACHMENT part 1.2 application/pgp-signature 
     
     >
     
      _______________________________________________
     
     >
     
      crossfire-devel mailing list
     
     >
     
     
      crossfire-devel at lists.real-time.com
      
      
     >
     
     
     
     https://mailman.real-time.com/mailman/listinfo/crossfire-devel
     
     
>
     
     
     


		
_______________________________
Do you Yahoo!?
Express yourself with Y! Messenger! Free. Download now. 
     
     http://messenger.yahoo.com
     
     

_______________________________________________
crossfire-devel mailing list
     
     crossfire-devel at lists.real-time.com
     
     
     https://mailman.real-time.com/mailman/listinfo/crossfire-devel
     
     
    


More information about the crossfire mailing list