[crossfire] Proposal: retire MAX_OBJECTS and object allocator

[Kevin Zheng]

Thanks, Mark, for the review.

On 3/10/20 9:56 PM, Mark Wedel wrote:
>> The object allocator allocates OBJ_EXPAND objects at a time and manages
>> its own free list. It is faster at allocating many objects at once, but
>> defeats memory debuggers and mitigation techniques. It also does not
>> return unused memory to the operating system.
> 
>  Not sure if still the case, but at one time, performance of allocating
> thousands of new objects individually increased lag (when loading new
> maps).  So doing it in blocks of 500 or whatever was faster.  At one
> point I had put in option to do individually allocation, simply for the
> reason above - if using a memory debugger, that block of objects failed
> to work.

I have not measured the latency loading maps. I'll measure this and
hopefully get back with some results.

>  Minor nit - even free() on a memory object does not return it to the OS
> - it tells malloc that block of memory is available again, and on a
> future call, malloc will re-use it, though you can get fragmentation
> (malloc may decide to use that memory for something smaller than an
> object).
This may be true for some allocators, but not jemalloc. Now, not all
allocators are jemalloc and that might be important to consider.

In jemalloc, unused dirty pages are kept around for re-allocation but
eventually "returned" to the operating system via madvise(...MADV_FREE).
top confirms that the server's resident memory can drop below its peak
resident memory when no swapping takes place. The jemalloc statistics
show the number of madvise() calls.

Linux appears to have gained MADV_FREE since 4.5, but I'm not sure if
the standard library allocator uses it.

>> object_free2() adds freed objects to a free list that is reclaimed at
>> the end of each server tick. It cannot immediately free object memory,
>> because some callers expect to be able to query it for FLAG_FREED to
>> detect object removal.
> 
>  querying for flag freed was always a bit buggy (it sort of relies that
> the given object won't get re-used, which works because the program is
> singled threaded and these freed objects will be last to be allocated
> when new objects are needed again).
> 
>  Also, as noted above, since memory is not returned to the OS, only the
> allocator, having crossfire itself keep a freelist makes perfect sense -
> it is going to need those objects again at some point, and objects are
> the main use of dynamic memory.   If the program did something like was
> in one state where it needed a bunch of malloc'd items, and did
> something with them, no longer needed them, but then needed a large
> amount of memory for a different object type, freeing them would make
> sense, because in that second allocation, it could re-use that freed up
> space.

I suppose this boils down to how much slower calloc() is compared to
grabbing from the internal freelist, and whether free() returns memory
on enough supported operating systems or not.

-- 
Kevin Zheng
kevinz5000 at gmail.com | kevinz at berkeley.edu
XMPP: kevinz at eecs.berkeley.edu