ICE regarding Allocator API

[Sp00ph]

I got an ICE while doing cargo t --features nightly with this:

Code snippets

lib.rs

#![cfg_attr(not(test), no_std)]
#![cfg_attr(feature = "nightly", feature(allocator_api, maybe_uninit_extra, maybe_uninit_uninit_array, new_uninit))]

extern crate alloc;

use core::mem::{forget, transmute_copy, MaybeUninit};
use core::ptr::{drop_in_place, slice_from_raw_parts_mut};

#[cfg_attr(not(feature="nightly"), path="stable.rs")]
#[cfg_attr(feature="nightly", path="nightly.rs")]
mod boxed;
pub use boxed::*;

#[inline]
pub(crate) fn init_slice<T, F: FnMut(usize) -> T>(s: &mut [MaybeUninit<T>], mut f: F) {
    struct Guard<T> {
        ptr: *mut T,
        len: usize,
    }

    impl<T> Drop for Guard<T> {
        fn drop(&mut self) {
            // SAFETY: It is guaranteed that exactly `self.len` items in the slice have been initialized.
            // `self.ptr` is guaranteed to be valid, even if `T` is a ZST, because it has been passed in
            // through the slice `s`.
            unsafe { drop_in_place(slice_from_raw_parts_mut(self.ptr, self.len)) };
        }
    }

    let mut guard = Guard {
        ptr: s.as_mut_ptr() as *mut T,
        len: 0,
    };

    for (i, a) in s.iter_mut().enumerate() {
        // Dropping a `MaybeUninit<T>` does nothing, so assigning to it like this
        // does not cause any memory unsafety issues.
        *a = MaybeUninit::new(f(i));
        guard.len += 1;
    }

    forget(guard);
}

nightly.rs

use crate::init_slice;
use alloc::{
    alloc::{Allocator, Global},
    boxed::Box,
};
use core::mem::MaybeUninit;

/// Initialize a dynamically-sized heap-allocated slice.
///
/// This function takes in the length of the returned slice as well as a function, which can use the index in the array to compute
/// the value for the item at that index. The function needs to implement [`FnMut`], which means it can also carry internal mutable
/// state which persists for all items.
///
/// # Examples
///
/// ```
/// #![feature(allocator_api)]
/// use init_array::init_boxed_slice_in;
/// use std::alloc::Global;
/// assert_eq!(&*init_boxed_slice_in(3, |_| 0, Global), &[0; 3]);
///
/// assert_eq!(&*init_boxed_slice_in(5, |i| i + 1, Global), &[1, 2, 3, 4, 5]);
///
/// let mut state = 0;
///
/// // arr[i] represents the sum of the first `i + 1` natural numbers.
/// let arr = init_boxed_slice_in(5, |i| {
///     state += i + 1;
///     state
/// }, Global);
/// assert_eq!(&*arr, &[1, 3, 6, 10, 15]);
/// ```
#[inline]
pub fn init_boxed_slice_in<T, F, A>(n: usize, f: F, alloc: A) -> Box<[T], A>
where
    F: FnMut(usize) -> T,
    A: Allocator
{
    let mut arr = Box::new_uninit_slice_in(n, &alloc);

    init_slice(&mut arr, f);

    // SAFETY: `init_slice` initialized the entire slice that is given to it, which in this case is the entire allocated slice.
    // Because all the items have been initialized, it's safe to transform it into the initialized slice by casting the pointer.
    unsafe { Box::from_raw_in(Box::into_raw(arr) as _, alloc) }
}

Meta

rustc --version --verbose:

rustc 1.54.0-nightly (657bc0188 2021-05-31)
binary: rustc
commit-hash: 657bc01888e6297257655585f9c475a0801db6d2
commit-date: 2021-05-31
host: x86_64-pc-windows-msvc
release: 1.54.0-nightly
LLVM version: 12.0.1

Error output

error: internal compiler error: /rustc/657bc01888e6297257655585f9c475a0801db6d2\compiler\rustc_codegen_ssa\src\mir\operand.rs:132:38: Deref of by-Ref operand OperandRef(Ref((%"std::boxed::Box<[std::mem::MaybeUninit<usize>], &std::alloc::Global>"*:  %6 = alloca %"std::boxed::Box<[std::mem::MaybeUninit<usize>], &std::alloc::Global>", align 8), None, Align { pow2: 3 
}) @ TyAndLayout { ty: std::boxed::Box<[std::mem::MaybeUninit<usize>], &std::alloc::Global>, layout: Layout { fields: Arbitrary { offsets: [Size { raw: 0 }, Size { raw: 16 }], memory_index: [0, 1] }, variants: Single { index: 0 }, abi: Aggregate { sized: true }, largest_niche: Some(Niche { offset: Size { raw: 0 }, scalar: Scalar { value: Pointer, valid_range: 1..=18446744073709551615 } }), align: AbiAndPrefAlign { abi: Align { pow2: 3 }, pref: Align { pow2: 3 } }, size: Size { raw: 24 } } })

Backtrace

When running cargo t --features nightly with RUST_BACKTRACE=1, no backtrace appears. When running it with
RUST_BACKTRACE=full, I get the following backtrace:

stack backtrace:
   0:     0x7ff830c2645f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h64b9092317105df4
   1:     0x7ff830c4f6aa - core::fmt::write::h4c8edbe553f89713
   2:     0x7ff830c199e8 - <std::io::IoSlice as core::fmt::Debug>::fmt::h97785632734c4dcf
   3:     0x7ff830c2a626 - std::panicking::take_hook::h12edddce955e43f1
   4:     0x7ff830c2a109 - std::panicking::take_hook::h12edddce955e43f1
   5:     0x7fffe9d29fa7 - rustc_driver::report_ice::h3049cb25589a84c3
   6:     0x7ff830c2ae79 - std::panicking::rust_panic_with_hook::hb792f6deccc56d02
   7:     0x7fffedefecd0 - <rustc_errors::snippet::Style as core::fmt::Debug>::fmt::hb979f8072521b87e
   8:     0x7fffedefec49 - <rustc_errors::snippet::Style as core::fmt::Debug>::fmt::hb979f8072521b87e
   9:     0x7fffee1a5681 - rustc_query_system::query::job::report_cycle::h81fc5a257d0f479e
  10:     0x7fffedf2b370 - rustc_errors::annotate_snippet_emitter_writer::AnnotateSnippetEmitterWriter::ui_testing::h625a176cbb1ed0cb
  11:     0x7fffedf31d25 - rustc_errors::HandlerInner::err_count::hafc992165e552e1d
  12:     0x7fffedf2f822 - rustc_errors::Handler::bug::hff657bce9edf0b10
  13:     0x7fffeddf30c8 - rustc_middle::ty::sty::<impl rustc_middle::ty::list::List<rustc_middle::ty::sty::Binder<rustc_middle::ty::sty::ExistentialPredicate>>>::principal_def_id::h4203bcb716d2851d
  14:     0x7fffeddec240 - rustc_middle::ty::subst::<impl rustc_middle::ty::list::List<rustc_middle::ty::subst::GenericArg>>::truncate_to::h5faae0127e315250
  15:     0x7fffeddec1e8 - rustc_middle::ty::subst::<impl rustc_middle::ty::list::List<rustc_middle::ty::subst::GenericArg>>::truncate_to::h5faae0127e315250
  16:     0x7fffeddf2ff9 - rustc_middle::ty::sty::<impl rustc_middle::ty::list::List<rustc_middle::ty::sty::Binder<rustc_middle::ty::sty::ExistentialPredicate>>>::principal_def_id::h4203bcb716d2851d
  17:     0x7fffee1a18c7 - rustc_middle::util::bug::bug_fmt::h9bfd4648b61d87bb
  18:     0x7fffea008d95 - rustc_codegen_llvm::llvm_::archive_ro::Child::data::hc789694af6654b76
  19:     0x7fffea00a466 - rustc_codegen_llvm::llvm_::archive_ro::Child::data::hc789694af6654b76
  20:     0x7fffe9ffe9a9 - rustc_codegen_llvm::llvm_::archive_ro::Child::data::hc789694af6654b76
  21:     0x7fffe9f8ab42 - rustc_codegen_llvm::type_::<impl rustc_codegen_ssa::traits::type_::LayoutTypeMethods for rustc_codegen_llvm::context::CodegenCx>::reg_backend_type::h895b44fdd8dd0179
  22:     0x7fffe9fc8e1c - <rustc_codegen_llvm::llvm_::ffi::PassKind as core::fmt::Debug>::fmt::hf964a257e2f76c5e
  23:     0x7fffe9ffadff - <rustc_codegen_llvm::base::ValueIter as core::iter::traits::iterator::Iterator>::next::h961b1947c8d7e5f7
  24:     0x7fffe9f8ca07 - rustc_codegen_llvm::type_::<impl rustc_codegen_ssa::traits::type_::LayoutTypeMethods for rustc_codegen_llvm::context::CodegenCx>::reg_backend_type::h895b44fdd8dd0179
  25:     0x7fffe9ffa711 - <rustc_codegen_llvm::base::ValueIter as core::iter::traits::iterator::Iterator>::next::h961b1947c8d7e5f7
  26:     0x7fffe9fa6201 - <rustc_codegen_llvm::LlvmCodegenBackend as rustc_codegen_ssa::traits::backend::CodegenBackend>::codegen_crate::h2ae70781e72ad261
  27:     0x7fffe9e5ed3f - rustc_interface::passes::BoxedResolver::to_resolver_outputs::h0d1350e5e7985bf3
  28:     0x7fffe9e810c6 - rustc_interface::queries::Queries::ongoing_codegen::h0e9f2e74141d6805
  29:     0x7fffe9d767dd - <rustc_driver::args::Error as core::fmt::Debug>::fmt::h37b23c9fa6ec772d
  30:     0x7fffe9d44841 - rustc_driver::pretty::print_after_hir_lowering::h3093e466fb72c6e2
  31:     0x7fffe9d77869 - <rustc_driver::args::Error as core::fmt::Debug>::fmt::h37b23c9fa6ec772d
  32:     0x7fffe9d45bfb - rustc_driver::pretty::print_after_hir_lowering::h3093e466fb72c6e2
  33:     0x7fffe9d96c81 - <rustc_driver::args::Error as core::fmt::Debug>::fmt::h37b23c9fa6ec772d
  34:     0x7fffe9d2d9ed - <rustc_driver::Compilation as core::fmt::Debug>::fmt::hb0ab8f97bd4d8039
  35:     0x7ff830c38c3c - std::sys::windows::thread::Thread::new::h7f0299499f002b12
  36:     0x7ff880be7034 - BaseThreadInitThunk
  37:     0x7ff881b82651 - RtlUserThreadStart

[hellow554]

The code doesn't ICE for me. Can you provide a repository which will lead to the error you described with all the necessary files, e.g. I just want to do a git clone ... and then use cargo test --features nightly and see that ICE. Can you do it for us? :)

[Sp00ph]

Sure, here you go

[Sp00ph]

Ok so I noticed a couple things:

• There were some pretty obvious bugs in the code, nothing that should've caused an ICE though (In fact the buggy functions passed their tests).
• Somehow the ICE went away when using Box::into_raw_with_allocator instead of just Box::into_raw:

// doesn't work
let b = Box::new_in(MaybeUninit::uninit(), &alloc);
// ... initialize b
return Box::from_raw_in(Box::into_raw(b) as _, alloc);

// this works
let b = Box::new_in(MaybeUninit::uninit(), alloc);
// ... initialize b
let (ptr, alloc) = Box::into_raw_with_allocator(b);
return Box::from_raw_in(ptr as _, alloc);

Of course this is only pseudo code, but if I'm not mistaken, the first version should work too.

[hellow554]

Unfortunatly this still doesn't ICE for me :( might be a platform specific problem? I see that you're using windows. I try to reproduce this tomorrow.

Cannot produce under linux with x86_64-pc-windows-gnu target, so this is probably a MSVC specific problem as well!

@rustbot modify labels: O-Windows O-windows-msvc A-allocators

[Sp00ph]

It's important to note that i got it working on the newest commit. To reproduce the ICE you'll have to use the initial commit, not the latest one. Should have maybe mentioned that in my last comment

[hellow554]

Should have maybe mentioned that in my last comment

You really should have :D
please update your post with the specific commit. I'm able to reproduce your ICE now. Thanks.

@rustbot modify labels -O-windows -O-windows-msvc

[Sp00ph]

Alright I updated the link. Sorry for the inconvenience 😅

[hellow554]

Thank you! This is probably a dup of #81270, but with your help I was able to find a very small program that leads to the exact ICE the other person had, which is fantastic, because in the other issue, the example was much larger. I think this can be safely closed therefore.

Thank you for reporting it, I highly appreciate it!

[Sp00ph]

Alright I'll close this issue then :)