[crossfire] Archetypes collection changes proposal

[Kevin Zheng]

On 9/16/20 2:38 AM, Nicolas Weeger wrote:
> To simplify, I'll call "assets" everything that is in the "arch" repository: 
> archetypes, pictures, faces, animations, treasures, artifacts, and what ever 
> else I'm missing ;)

Do maps have a place here, as well?

> The goal: make it easier to add or replace existing assets for contributors, 
> simplify assets distribution, and make it easy for server admins to add/remove 
> extra or "official" assets.

These sound like good goals to me.

> My proposal:
> - have the server recursively browse "data" directory to find all assets
> - browse in a deterministic manner (sorted case-sensitive, depth first)
> - allow override of defined assets (thus the need to have a deterministic 
> browsing pattern)
> - remove the requirement for "collect"
> - keep the "collect" mechanism to consolidate assets in big files easy for 
> distribution - have it work for non official assets too

To keep things simple, why keep the collect mechanism? Couldn't we put
the "official" assets into a big ZIP file, and teach the server how to
open ZIP files like ordinary directories? Scan the index, create sorted
case-sensitive depth-first order, etc.

> One big drawback is that the server startup will probably be slower, and the 
> code more complex (to handle overriding).
> 
> I think that's an acceptable drawback, but that's subject to discussion ;)

`make do-collect` on a cold cache on my dated computer, with an SSD,
takes 6.3 sec. I imagine it could be a bit slower for those with
spinning magnetic media. This seems acceptable for a video game, and
particularly for a server that's supposed to be long-running. But, we
should still profile, see what's slow, and try to make this faster.

What would your strategy be for a change like this? One interim
strategy, just to keep everything working, could be to invoke the
current collect process at server start via fork/exec, read in output
via loader, then start. Then merge the steps until they become one.

Regards,
Kevin