unknown Bug/NotBug about span benchmarks

[rootflood]

this issue on .net runtime

Description

i don't know this is a bug or not, recently i saw an issue about performance impact in spans: #54672
i use that benchmark but i got unexpected strange result, i use a class as base of benchmark and other classes as sub benchmark. there isn't no benchmark in sub classes and they are just for test something which was strange for me and the benchmark was only in base class. when i test same method then results was different but i don't use any method in sub classes. i don't know what is the problem here and come from my codes or not because i'm not professional at this field.

the bench repo

Configuration

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.985 (21H1/May2021Update)
.NET SDK=5.0.100
[Host] : .NET 5.0.0 (5.0.20.51904), X64 RyuJIT
DefaultJob : .NET 5.0.0 (5.0.20.51904), X64 RyuJIT

Benchmark

    public abstract class BenchmarksBase
    {
        protected int[] _array;

        [Params(1_000_000)]
        public int Count { get; set; }

        [GlobalSetup]
        public void GlobalSetup()
        {
            _array = new int[Count];
        }
        
        [Benchmark]
        public int Span_ForEach()
        {
            var Res = 0;
            var source = _array.AsSpan();
            for (int i = 0; i < 1000; i++)
            {
                Res = 0;
                foreach (var item in source)
                    Res += item;
            }
            return Res;
        }
    }

Sub Benchmarks

    public class Benchmarks1 : BenchmarksBase
    {}
    public class Benchmarks2 : BenchmarksBase
    {
        public int Array_For()
        {
            var array = _array;
            var Res = 0;
            for (int To = 0; To < 1000; To++)
            {
                Res = 0;
                for (int i = 0; i < array.Length; i++)
                    Res += array[i];
            }
            return Res;
        }
    }

    public class Benchmarks3 : BenchmarksBase
    {
        public int Array_For()
        {...} // same like Benchmarks2.Array_For()

        public int Array_For1()
        {...} // same like Benchmarks2.Array_For()

        public int Array_For2()
        {...} // same like Benchmarks2.Array_For()

        public int Array_For3()
        {...} // same like Benchmarks2.Array_For()
    }

Benchmark Results

Benchmarks1 class:

Method	Count	Mean	Error	StdDev	Gen 0	Gen 1	Gen 2	Allocated	Code Size
Span_ForEach	1000000	736.6 ms	10.36 ms	11.52 ms	-	-	-	-	65 B

Benchmarks2 class:

Method	Count	Mean	Error	StdDev	Code Size	Gen 0	Gen 1	Gen 2	Allocated
Span_ForEach	1000000	1.054 s	0.0064 s	0.0057 s	65 B	-	-	-	-

Benchmarks3 class:

Method	Count	Mean	Error	StdDev	Code Size	Gen 0	Gen 1	Gen 2	Allocated
Span_ForEach	1000000	733.8 ms	8.68 ms	7.25 ms	0 KB	-	-	-	1 KB

[adamsitnik]

Hi @rootflood

You are most probably hitting an issue with memory alignment.

You can use MemoryRandomization feature to enforce BDN to re-allocate the buffers allocated in [GlobalSetup] for every iteration or use the new NativeMemory.AllocAlignedMemory API introduced in .NET 6 (you can download latest bits from here) to allocate aligned memory.

More info related to alignment:

• Enforce code alignment #756
• Make memory alignment more random #1513
• Memory Randomization performance#1602

Since there is nothing more we can do, I am closing the issue. Thanks!