[crossfire] crossfire image handling

[Mark Wedel]

  I'm writing this message to mostly capture ideas/thoughts from last nights IRC 
discussion before they leave my brain.

  The basic idea is to communicate all face (and I think animation data) to the 
client before play starts.  This should improve bandwidth some (no spike in sent 
data when entering a new map to send images, etc).  This also simplifies a lot 
of code - no need to check to see if images have been sent - this is more 
relevant if things like embedding images is added.

  To do this would mean that all clients must do image caching.  A problem right 
now is that to just download information on the 4000 images in use takes 
measurable amount of time.

  To fix this, the image numbers on a server would remain constant - new or 
changed images would get added to the end.  Right now, all images are put in 
alphabetical order, and then assigned numerically.  Thus, if a new image 
starting with A is added, all images after that point have new numbers.

  So under this scheme, if the server currently had 4126 images, and some new 
images were added, they would get numbers 4127, 4128, and so on - in this way, 
images 1-4126 remain the same, so the the client doesn't need to request names 
and checksum information for those.  Likewise, if an image changes, its old 
entry would get blanked out (a whole in the image numbers), and it put at the end.

  In this way, when the client connects, it request the highest image number the 
server has and records that away.  If its stored value does not match the number 
the server reports, it knows from what point to start requesting that new 
information - so in general it would likely need a fairly small set of new 
images, and if connecting to the same server, in many cases it would know 
instantly that it is up to date.

  For any new images, it could then use askface to fetch them.  It might be nice 
to also allow out of band methods (ftp, http) as an alternative to fetch images 
- in this case, it would presumably be fetching things the consolidated image 
files.  However, this is more complicated - if only 10 images have changed, I 
don't want to fetch an image file with 4000 images, so incremental image files 
would be needed.  But if the collect script knows which images have changed or 
are new, it is a pretty simple matter to put those images into incremental image 
files.

  Other thoughts:
I've mulled over the idea of using checksums (md5 or something) instead of the 
current checksum and image name.  With this, the server would basically just say 
'image 6 checksum is 12ab53....'.  The client can then check to see if has an 
image of that checksum anyplace - if doesn't care where it got it from 
(different server, part of standard image package, etc), and if so, uses it, 
otherwise requests it.

  In some ways, this can be more efficient, because some images have long names. 
  It looks like the current average name length is 12 characters - an md5 would 
be 16 bytes, but would no longer be needed to send checksum data since that is 
the checksum, so would be a wash.

  The problem here is that it is likely we will want to be able include image 
names in text messages and the like, so we can't get away from names.

  I'm not sure how good our current checksum method is - it is a 32 bit, but I 
doubt is especially collision resistant.

  A current problem with just storing images by image names is you get the case 
of different servers may have an image by the same name but different contents. 
  This basically means that caching information must be stored per server. 
While in most cases, these duplicate names are not likely to cause much issue, 
one could imagine that the names are such that they do cause issues (calling it 
something like map.111 which is a map meant to display to players as part of a 
quest).

  Using good checksums means that the client can basically share images from all 
the servers it may play against, but we avoid the problem of images of same name.

  One other random thought:  The new login method will be such that no map/image 
data will be shown until the player logs in and selects a character.  What this 
means is that the client could conceivably do a lot of the image updating while 
that is happening, and only if not done by the time the login is finished does 
it have to pop up a notice saying something like 'fetching images'