[CF-Devel] reduction of data when casting spells

[Mark Wedel]

 I'll try to sum up the points, and limit quoting to a minimum.

 First, trying to be more intelligent about what we send to the client is
actually more work for the server (although less bandwidth). This is because the
server now needs to go through and try to find duplicates.

 Second, I believe slowness on the server is generally caused by processing lots
of spell objects on the same space internally, and not bandwidth to the client. 
This is because any time a new spell object is added to the space, it has to
look at all the other spell objects to see if might affect them.  This basically
means that if you have 50 spell objects, and another gets added, it needs to
examine 51 objects.  And to get to having 50 spell objects, it will have had to
do 1275 examines.

 This can only really be fixed by trying to limit use of lots of spell casting
monsters on the same map, and a complete re-design of how this processing
happens.  I've given some thoughts to this, and all and all, its not a simple
matter.

 It would be an interesting experiment to change the nrof 0 to nrof 1 in all the
spell effect objects and just see what happens in terms of how many objects may
actually get merged.  I know that doing this will almost certainly break other
places (as I don't think the spell effect propogation is currently intelligent
enough to look at the nrof field to know how many it should put on the next
space, and I also bet the damage function doesn't look at nrof either).  I have
a feeling there may be some amount of merging, but not enough to make a real
difference.

 PC's scheme of identifying spell effects and counting the number to only send
one with a proper nrof would work, but gets sort of complicated.  First, the
code has to know what all the potential effects are (might just presume that
flag_flying is the right thing, or no_pick or the like) - the code certainly has
to be smart enough so that it doesn't do that to a bunch of different arrow
types.  It is also tricky in that whenever we run into a spell effect, we either
need to record the effect type (maybe by looking at the arch it points to, or
name?), and then record someplace we've sent that effect type.  This does mean
that whenever we get another affect, we need to look at that record to see if
its already been sent - that could be costly.  And alternative would be to have
temporary flag that we clear on all objects in the space, and then when we get a
spell effect, look through the rest of the objects on that space setting the
flag on matches - when the processing happens, if that flag is set, it knows not
to send it.  Proper clearing of that flag would be a must for things to work
properly.

 Note that the knowing how many spell objects of that type needs to be
determined when we get the first object of that type - to try and go back
through the packet structure and change that value after the fact would be a
real pain.

 For number of objects in the look window - my thought is to make that dynamic
at some point.  I didn't do that in my implementation as that would add a bit of
complication I didn't want to add at that point.  In terms of dynamic, I was
meaning dynamic player settable (and not something the server figures out). 
This, when your going into the dungeon to clear out all the monsters, you
probably have that set to a low value and use pickup mode 10 or the like which
won't care about that setting.  Then when you go back in vacuum mode, you may
set that to a higher value to better examine the stacks.

 Gradually sending more items each tick would be fairly tricky, and the number
sent per tick would have to be settable (if on a fast link, you probably want a
whole bunch, if a slow link, just a few).  I would prefer to make the number of
objects viewable in the look window settable - easier to do. 

 As previously said, the handling of sending of data to the look window probably
needs to get redone for 2.0. 

 Adjusting the amount of data based on size of pending output might be possible
- first, getting the size of the OS pending output buffer would be needed.  But
once again, this would need to be settable - if your playing on a fast link, a
lot of data may get sent to you in a tick, and thus a large ouptut buffer, but
it will get cleared very quickly (remember, the server basically forms all the
data it will send to the client at the same time, and then lets the OS sending
that over the intervening time until the next tick).

 But also, crossfire does try to be intelligent in caching in some regards.  For
example, for the map, it only sends what has changed.  So if we skip sending the
map for a tick or two, chances are that then when we finally get around to
sending it, it will have many more changes (OTOH, we do avoid the network
overhead).  But its also tricky on knowing how to catch up on some of these
changes.

 For example, data is backlogged a bit, and the player kills the monster, and
moves onto the space with its treasure.  Server sees this backlog, and decides
not to send the look window (which is probably the biggest hog in terms of
bandwidth).  Server does need to know to try to resend that later.  IT may work
on the fact that we haven't cleared the flag - I'd actually have to think about
that further.

 Its also unclear what can miss a tick.  Can you afford to miss an stat update
until the connection is speedy?  If your hp has just dropped from to 10, you
probably want to get that information ASAP, even if it means going on the back
of a decent sized backlog.