The memchr implementation has UB

[carbotaniuman]

https://stackoverflow.com/questions/47315902/is-it-legal-to-call-memchr-with-a-too-long-length-if-you-know-the-character-wil

Basically a memchr call with too big a size is valid if you can guarantee that it will not be reached. Given mustang is not yet intended to be super performant, we can probably just replace this with a simple loop

Opening for storage after a discussion on the Rust community discord.

[sunfishcode]

Good spot! It's unfortunate, because the memchr crate has some nice optimizations, but it appears we'll have to stop using it.

[bjorn3]

Does the memchr crate speculatively read across pages? If not, ensuring that the memchr C function can't ever be inlined should be enough to prevent the compiler from exploiting UB even if it still technically exists.

[sunfishcode]

At a brief glance, I don't see any obvious unaligned loads in memchr's optimized routines, so it would probably work in practice.

On the other hand, the public documentation of the memchr crate doesn't guarantee that it doesn't do these things. Also, inline(never) is not a complete optimization barrier. Also, tools like miri can see these kinds of problems even when optimizers can't.

I don't yet know who will want to use mustang or c-scape, or what they'll want to use it for. In the fullness of time, we could implement the optimizations that the memchr crate does in a way that respect the C memchr semantics. But until then, will optimization be more important to them than theoretical correctness? I really don't know. But my guess is theoretical correctness.

That said, I am aware of an outstanding theoretical correctness bug in c-scape: #142. I submitted #143 which fixes the easy cases, and am thinking about having c-scape use a stack-allocated buffer in read so that it can write into the user's buffer with copy_nonoverlapping in order to avoid the UB. That's a little heavy-handed; bytecodealliance/rustix#81 tracks fixing this properly, which might depend on how rust-lang/rust#78485 turns out.