[crossfire] crossfire image handling

Mark Wedel mwedel at sonic.net
Sun Jan 31 23:45:31 CST 2010


On 01/31/10 04:40 AM, Nicolas Weeger wrote:
>>    The basic idea is to communicate all face (and I think animation data) to
>> the client before play starts.  This should improve bandwidth some (no
>> spike in sent data when entering a new map to send images, etc).  This also
>> simplifies a lot of code - no need to check to see if images have been sent
>> - this is more relevant if things like embedding images is added.
>>
>>    To do this would mean that all clients must do image caching.  A problem
>> right now is that to just download information on the 4000 images in use
>> takes measurable amount of time.
>>
>>    To fix this, the image numbers on a server would remain constant - new or
>> changed images would get added to the end.  Right now, all images are put
>> in alphabetical order, and then assigned numerically.  Thus, if a new image
>> starting with A is added, all images after that point have new numbers.
>
> I think that should implemented in a way coherent with the expected image
> update frequency.
>
> If we expect pics to be updated really often, then it makes sense to only send
> changes.
> If on the other hand pics don't change that often, then isn't the current
> method ok (assuming clients download all pics at start)?

  We don't really know how often images will change, so from that point, hard to 
say what is the right approach.

  If image updates are infrequent, then the client could download the image 
information once and keep using it.  However, if there is any change, no matter 
how small, the client then needs to download the entire image name->number 
mapping, which at current time is about 5000 images.

  Of course, that number is only likely to increase, which means that for 
additions, we end up download a lot of data that we don't need to.


>>    So under this scheme, if the server currently had 4126 images, and some
>> new images were added, they would get numbers 4127, 4128, and so on - in
>> this way, images 1-4126 remain the same, so the the client doesn't need to
>> request names and checksum information for those.  Likewise, if an image
>> changes, its old entry would get blanked out (a whole in the image
>> numbers), and it put at the end.
>
> But then the server itself has to track image changes, which isn't that easy
> with current implementation.

  Really it is the collect script that would do this, not the server.  In fact, 
the server wouldn't care - all that would be needed for the server (I believe) 
is for it to be able to handle bmaps files that are not in alphabetical order.

  For the collect script, what it would need to do is read in the existing bmaps 
file, and whenever it runs into a face, see if it is an existing face or new one 
- shouldn't be that hard.  The one change here is that the bmaps file would need 
to contain checksums so the collect script could detect changed faces, but even 
that isn't that hard.


>>    In this way, when the client connects, it request the highest image
>> number the server has and records that away.  If its stored value does not
>> match the number the server reports, it knows from what point to start
>> requesting that new information - so in general it would likely need a
>> fairly small set of new images, and if connecting to the same server, in
>> many cases it would know instantly that it is up to date.
>
> What about using a timestamp for the last change?

  Maybe, but that doesn't really solve the problem of needing to download 5000 
faces to find information about one new one.

  The other issue with that is at least right now, the collect script will 
always write out a bmaps file when run, even if no changes.  So if you make a 
change to an archetype, run collect, that file is updated, so from that 
perspective, timestamp isn't great.


>>    I'm not sure how good our current checksum method is - it is a 32 bit,
>> but I doubt is especially collision resistant.
>
> Right now doesn't matter, as name is taken into account (I think), client
> caches from the name.

  It does.  But if checksum was good (non collision resistant), checksum could 
be used to see if client already has that image.

  A problem right now with the C client (maybe it was fixed) was the fact it 
uses image names causes problems when different servers use different images for 
the same face.

  I connect to server A, and it downloads grass.111
  I connect to server B, it uses a different image, so downloads grass.111
  I go back to A, since that file has been replaced, it downloads again.

  Now that is really an implementation detail - one way to handle it is per 
server image caches (so client has a different directory to cache A vs B 
images).  But that is sort of wasteful when 90% may be the same.  Ideally, if 
image is the same, you want to share them, but if different, you want to be able 
to store them away and access them.

  Another more real situation could be case of image changing on server then 
reverted for whatever reason.  In that case, once again, you'd like to just be 
able to go back to the original and not download again.

  This is all really a client issue, but if the checksum was really good, it 
does make life easier.


>>    One other random thought:  The new login method will be such that no
>> map/image data will be shown until the player logs in and selects a
>> character.  What this means is that the client could conceivably do a lot
>> of the image updating while that is happening, and only if not done by the
>> time the login is finished does it have to pop up a notice saying something
>> like 'fetching images'
>
>
> Depending on the expected transfer time, obviously.
> I remember GTK client having all image from the start, though (crossfire.0?
> something like that).

  Yes - prebuilt image libraries can be included - and if we go to forced cache, 
would no longer be an option but a requirement.

  This removes the need to download a lot of images, but doesn't remove the need 
to know the number to name mapping.




More information about the crossfire mailing list