Make try_init_allocator() a ctor

[TerrorJack]

We can make try_init_allocator() a ctor, and even avoid calling it in dlmalloc(). Assigning the ctor with highest priority will enable regular ctors to call malloc() safely, and wizer-like optimization will be capable to strip out some extra bits.

[sbc100]

The entire range of 0-100 I believe is reserved for system functions like this. I'm not convinced 0 is the best choice here, but its not not hard to change in the future.

In emscripten we document the priorities in order to keep track of all the order which this stuff happens: https://github.com/emscripten-core/emscripten/blob/main/system/lib/README.md?plain=1#L9-L28

[sunfishcode]

Practically speaking, right now whenever we add a new constructor in wasi-libc, we just grep for all the constructors in the wasi-libc tree to figure out the ordering constraints, which seems ok for now, as there aren't very many.

[TerrorJack]

I believe assigning priority 0 is a correct choice, given:

• dlmalloc is a service which is likely used by other ctors
• try_init_allocator itself doesn't rely on other ctors to be run before

We can also have a documentation somewhere to list all our ctors. I can add one in this PR if you folks think it's appropriate, though not sure where's the best place to add it.

[sbc100]

I think its ok to simply use grep as Dan says, I was just pointing out that it can get complicated, and that one needs to be careful when choosing the order of these things, once there is more than one.

There are also some things that might (one day) need to happen before malloc initialization (e.g. TLS setup, application of relocations in PIC code) although I don't think any of those apply yet in wasi-libc.

[sunfishcode]

I think this makes sense. We often like to avoid static ctors because they often slow down startup times due to doing extra work up front, however in this case, most programs are calling malloc early in startup anyway already, so the extra work needs to be done anyway.

[sbc100]

Out of interest, why does lazy-initialization inhibit "wizer-like optimization"?

[sunfishcode]

The static call graph has dlmalloc calling try_init_allocator, so try_init_allocator can't be easily deleted. If we remove that call and make it a constructor, then wizer can delete it after it runs.

[sbc100]

The static call graph has dlmalloc calling try_init_allocator, so try_init_allocator can't be easily deleted. If we remove that call and make it a constructor, then wizer can delete it after it runs.

Nice. That makes sense now.

Presumably in such cases dlmalloc iself would persist, which I assume would massively dominate the size of try_init_allocator? But I guess we care about every byte.

[sunfishcode]

Yeah, the code size win there likely isn't huge, but it's something. Likewise the win of not doing the load+branch in every malloc call is likely not huge, but it's something.

[sunfishcode]

I am now thinking about how this PR interacts with upcoming use cases for wasm which involve calling malloc etc. before user code runs. For example, in the component model canonical ABI, value imports aren't spec'd yet, but when they are, they're expected to work by having the outside call in to allocate space before calling the first user function. I don't think this is necessarily a show stopper, but I do want to think it through here.

Is the motivation for this PR to avoid the problem that the malloc init code doesn't know where __heap_base memory ends, and calls sbrk(0), assuming that no one else has done memory.grow before that point? In think we really do want to add a __heap_end symbol to mark the end to fix that problem, as that's fully robust in the presence of memory.grow happening at any time.

[sunfishcode]

I've now posted https://reviews.llvm.org/D136110 to add a __heap_end symbol.

[sbc100]

I am now thinking about how this PR interacts with upcoming use cases for wasm which involve calling malloc etc. before user code runs. For example, in the component model canonical ABI, value imports aren't spec'd yet, but when they are, they're expected to work by having the outside call in to allocate space before calling the first user function. I don't think this is necessarily a show stopper, but I do want to think it through here.

Is the motivation for this PR to avoid the problem that the malloc init code doesn't know where __heap_base memory ends, and calls sbrk(0), assuming that no one else has done memory.grow before that point? In think we really do want to add a __heap_end symbol to mark the end to fix that problem, as that's fully robust in the presence of memory.grow happening at any time.

Are you interested in knowing the original heap end that was set at compile time, or the current size of the memory? (I guess its the former since you can use an intrinsic to know that current memory size).

[sunfishcode]

Are you interested in knowing the original heap end that was set at compile time, or the current size of the memory? (I guess its the former since you can use an intrinsic to know that current memory size).

It's to know the original heap. The idea is, between the time when wasm-ld runs and the wasm malloc init code runs, someone may have used memory.grow to allocate memory. If that happens, the extra memory should be considered in-use, and malloc shouldn't clobber it. So malloc needs to know the extend of what wasm-ld allocated.

[TerrorJack]

Thanks for all the review comments! Closing this one since #377 is moving in the right direction.