[crossfire] Handling massive loot

[Preston Crow]

On 2021-03-23 00:27, Mark Wedel wrote:
> On 3/22/21 11:58 AM, Nicolas Weeger wrote:
>> Hello.
>>
>> One issue that happens is server delays when dealing with big loot.
>>
>> For instance a player selling some thousands rods can stuck the 
>> server for
>> some seconds... Or even dropping many items on a big pile can be 
>> quite slow.
>>
>>
>> So to handle that correctly, I was thinking of changing the server 
>> code like
>> that:
>>
>> - split data/commands reception and processing - have one loop that 
>> reads from
>> the socket, put commands in a waiting queue, then another function later
>> processes those commands
>>
>> - for commands like pick/drop/examine, enable them to work on multiple
>> processing loops instead of a single. So for instance the command 
>> would drop
>> 100 items, then store its status, put itself back on top of the command
>> waiting queue, and exit - this way the server can process something 
>> else.
>
> Being multi threaded would help some things out.
>
> It has been a while, by my recollection is the drop (and pickup) 
> problem is that these are in fact handled by the server, eg, 'drop 
> all' is done on the server, dropping everything of matching criteria.  
> And likewise, pickup all is done on the server, so slowing down the 
> command processing from the client doesn't help out.
>
> It is also my recollection in that the reason this is slow that each 
> time an object is moved (ground to player or vice versa), it has to 
> check the destination for duplicates to merge them (otherwise, you 
> could have 20 different entries for silver coins).  This becomes an 
> O(n^2) operation, so when dropping a few items, it is pretty fast, but 
> when dropping a stack of 100 different items, that takes a lot of time.
>
> There are a couple ways to deal with this:
> - A thought I had a while back is once a space gets too many items on 
> it, items fall over to neighboring spaces - this would reduce the 
> upper threshhold over that operation (never too many items to examine 
> on a space), but this could basically result in an entire shop being 
> filled to a level of 25 items deep.
> - For items going to the floor, marking the space as needing 'merging' 
> later on could be done (one could even imagine something like a shop 
> keeper standing on the space as this merging is done, preventing 
> players from interacting with it)
> - For items going into the players inventory (they step onto a pile of 
> junk), deferred merging could be done, but would result in new items 
> getting send to the client, and then and update down the road that the 
> new item does not exist and in fact item X has a different quantity.  
> I'm not sure how visually that works out.
> - Maybe limit the number of items picked up at once, even with pickup 
> all, to something like 25 or other number where performance is still 
> reasonable.  This makes it harder to clean up everything in the 
> dungeon, but is somewhat more realistic (you can only pick up stuff so 
> quickly).
> - I can't remember if the gold insertion when selling a bunch of items 
> is intelligent.  That is to say, if you sell 100 items, does is 
> calculate the net proceeds and inserts it once (intelligent) or 
> inserts the coinage 100 items (1/each item, keeping in mind that there 
> could actually be several different coins being inserted for each 
> item).  So changing that (deferred gold updating) could also have some 
> benefit.
>
Well, first look over the drop code to be sure there aren't any 
optimizations that have been missed, but then I would suggest looking 
for the simplest solution.  For example, only merge-match on the top 50 
items, but then flag the square as needing merging. Move the map save 
code to a separate thread, and have that thread do the merging.

Pickup is more complicated, as that can't be deferred.  And did the 
issue with loading an apartment map with too many items get fixed?

I really liked the general solution idea that someone mentioned years 
ago where when something is too slow, the server drops ticks on that map 
until it's caught up, but other maps continue to get processed.  The 
easiest way of doing something like that would be to have one thread per 
map, but that's probably excessive, and having tiled maps makes per-map 
processing more complicated.  A more general system would be to have 
multiple threads that pick up processing for maps, and maps would be put 
on a queue, so anything that makes things slow would only impact that 
map.  It gets a little complicated when things move from one map to 
another (especially with tiled maps), so perhaps active tiled maps would 
be put on the queue as a unit.