[crossfire] Map cache

[Brendan Lally]

On 8/26/05, tchize <
     
     tchize at myrealbox.com
     
     > wrote:
>
     
      Le Vendredi 26 Août 2005 13:40, Brendan Lally a écrit :
     
     >
     
      > On 8/26/05, tchize <
      
      tchize at myrealbox.com
      
      > wrote:
     
     >
     
      > > This isn't as simple, whole server code is thread unsafe, this mean
     
     >
     
      > > if you load map in a separate thread you will break about all likned lsits used in
     
     >
     
      > > server (objet pool, live objet lists, map lists, shared strings, and static
     
     >
     
      > > variables used in some functions).
     
     >
     
      >
     
     >
     
      > The way you would get around that would be to have all the required
     
     >
     
      > linked lists of items created for the map that is being loaded
     
     >
     
      > seperately, then every tick the main thread asks 'are you ready to
     
     >
     
      > merge' (checks the value of a variable) and when that is true, it
     
     >
     
      > would then call a function in the main thread that took the pointers
     
     >
     
      > for the objects in the sub-linked lists and inserted the whole lot
     
     >
     
      > into the main list in one go. If done properly, this isn't slower than
     
     >
     
      > 1 insertion, but is a lot quicker than 2500 (as is the case for the
     
     >
     
      > world maps)
     
     >
     
     
     >
     
      as i said, this is *not* as simple (eg object pool, you have to use it, and it's
     
     >
     
      access is not thread safe, the file parsing code is also not thread safe, so you
     
     >
     
      can only load one map / player file at a time). There are not only
     
     >
     
      linked list related to map, and map loading code does more
     
     >
     
      than just loading the file (create treasures a.s.o), so lots of server functions
     
     >
     
      are called during map load, and nearly 100% of them are not multithreadsafe.
     
      
The file accessing code would need rewritten a little, although, if
you allow one thread to do all file loading, then it doesn't need to
be done in parrallel, since the main thread will just ignore it, then
there would be the whole set of structs created for the map (this
includes treasure), where there would be a seperate set of linked
lists created. So instead of merely having FirstObject linking to all
the other objects, you have that, but also add TmpFirstObject which is
the start of that component of the linked list that corresponds to the
object on the loading map.

Effectively two object pools, a temporary one, and a permenent one,
and then you merge in the temporary pool once loading is finished.

Also you have a seperate list of shared strings, with their own ref
count, and a function to merge them with the main set in one
operation.

anyway, a rough look at the functions involved would suggest:

get_linked_map would become redundant, 
load_map_header would be fine (if we have 1 file loading thread, the
other thread won't be loading maps)
load_original_map would simple need to use the replacement for
get_linked_map, the main loop would need a link_map function to bring
the map and all its objects into the main set of linked lists.
the object code would need some changes, although mostly just to
ensure a seperate set of linked lists, and to stop it assuming that
everything is in the FirstObject list all the time, if a
get_tmp_object is written to replace get_object, and all other
functions use that, then mostly it should work.

The overlay code would probably break somewhat, but they are somewhat
buggy anyway, and fixing that would be something nice to do.

I also don't know how well the random map code would cope with all of
this (but only because I haven't looked).

Almost certainly there are some other things that I have missed, but
it does look like something that is somewhat short of a complete
rewrite.

In any event, there are going to be limits to what can be done, since
/any/ mechanism is going to fail when you are above more than 1 or
maybe two map loads a second with the current data structures being
used (ways around that might be to seperate the idea of object size
and face size, so that for instance, the entire floor of a building
might be one object, repeating the same face over and over again -
this would cause serious breakage everywhere though, including with
existing clients and multi-headed monsters).

Certainly though, I agree about the profiler point, if only to decide
when certain parts of the map loading should be done, I just don't
think that you /can/ make the map loading time less than about 1/10th
or so of a second, when you have too many objects there (at least
without changing a lot of the data types involved).