Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thread safety seems to be not guaranteed #2330

Open
LibeUltra opened this issue Feb 27, 2025 · 0 comments
Open

Thread safety seems to be not guaranteed #2330

LibeUltra opened this issue Feb 27, 2025 · 0 comments
Labels

Comments

@LibeUltra
Copy link

LibeUltra commented Feb 27, 2025

Describe the bug
Calling WriteRecord or NextRecord in a multithreaded scenario causes the error in .NET Framework 4.8:

System.IndexOutOfRangeException
  HResult=0x80131508
  Message=Probable I/O race condition detected while copying memory. The I/O package is not thread safe by default. In multithreaded applications, a stream must be accessed in a thread-safe way, such as a thread-safe wrapper returned by TextReader's or TextWriter's Synchronized methods. This also applies to classes like StreamWriter and StreamReader.
  Source=mscorlib
  StackTrace:
   at System.IO.FileStream.WriteFileNative(SafeFileHandle handle, Byte[] bytes, Int32 offset, Int32 count, NativeOverlapped* overlapped, Int32& hr)
   at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.Write(Char[] buffer, Int32 index, Int32 count)
   at CsvHelper.CsvWriter.FlushBuffer()
   at CsvHelper.CsvWriter.NextRecord()
   [...]

While in .NET 8 I get another error, not concurrency-related:

CsvHelper.WriterException
  HResult=0x80131500
  Message=An unexpected error occurred.
IWriter state:
   Row: 1
   Index: 0
   HeaderRecord:
1

  Source=CsvHelper
  StackTrace:
   at CsvHelper.CsvWriter.WriteRecord[T](T record)
   [...]

Inner Exception 1:
ArgumentException: An item with the same key has already been added. Key: System.Int32

To Reproduce
I used the same coded in both cases:

CsvConfiguration csvWriterConfig = new CsvConfiguration(CultureInfo.InvariantCulture) { Delimiter = ";" };
using StreamWriter writer = new StreamWriter("file.csv");
using CsvWriter csv = new CsvWriter(writer, csvWriterConfig);

Parallel.ForEach(Enumerable.Range(0, 10000), x =>
{
    csv.WriteRecord(x);
    csv.NextRecord();
});

Expected behavior
I excepted the data to be written concurrently to the CSV file.
I read about: #1623 but the last comment states that it is no longer a problem

Additional context
I'm trying to implement a pipeline with the library https://github.com/Open-NET-Libraries/Open.ChannelExtensions starting from an IAsyncEnumerable<T> of data, transforming each item creating a model class, and write those models into multiple CSV files batching by a number. Since I'm dealing with millions of items, I wanted to speed up things and write the data concurrently to the file. To lock asynchronously I use https://github.com/StephenClearyArchive/AsyncEx.Coordination

AsyncLock _evictionLock = new AsyncLock(); //Async version of lock
CsvHelperFileHandle csvFile = NewCsvFile(); //This is a wrapper of CsvWriter
int count = 0;
await data //IAsyncEnumerable<T>
    .ToChannel()
    .PipeAsync(maxConcurrency: 100, capacity: 1000, transform: CreateModelAsync)
    .ReadAllConcurrentlyAsync(maxConcurrency: 100, async model =>
    {
        Interlocked.Increment(ref _count);
        if (Interlocked.CompareExchange(ref _count, 0, BatchSize) == 0)
        {
            using (await _evictionLock.LockAsync().ConfigureAwait(false))
            {
                csvFile.Dispose();
                csvFile = NewCsvFile();
            }
        }

        csvFile.Csv.WriteRecord(model);
        csvFile.Csv.NextRecord();
    })
    .ConfigureAwait(false);
@LibeUltra LibeUltra added the bug label Feb 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant