You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Calling WriteRecord or NextRecord in a multithreaded scenario causes the error in .NET Framework 4.8:
System.IndexOutOfRangeException
HResult=0x80131508
Message=Probable I/O race condition detected while copying memory. The I/O package is not thread safe by default. In multithreaded applications, a stream must be accessed in a thread-safe way, such as a thread-safe wrapper returned by TextReader's or TextWriter's Synchronized methods. This also applies to classes like StreamWriter and StreamReader.
Source=mscorlib
StackTrace:
at System.IO.FileStream.WriteFileNative(SafeFileHandle handle, Byte[] bytes, Int32 offset, Int32 count, NativeOverlapped* overlapped, Int32& hr)
at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.FileStream.Write(Byte[] array, Int32 offset, Int32 count)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.IO.StreamWriter.Write(Char[] buffer, Int32 index, Int32 count)
at CsvHelper.CsvWriter.FlushBuffer()
at CsvHelper.CsvWriter.NextRecord()
[...]
While in .NET 8 I get another error, not concurrency-related:
CsvHelper.WriterException
HResult=0x80131500
Message=An unexpected error occurred.
IWriter state:
Row: 1
Index: 0
HeaderRecord:
1
Source=CsvHelper
StackTrace:
at CsvHelper.CsvWriter.WriteRecord[T](T record)
[...]
Inner Exception 1:
ArgumentException: An item with the same key has already been added. Key: System.Int32
To Reproduce
I used the same coded in both cases:
CsvConfigurationcsvWriterConfig=newCsvConfiguration(CultureInfo.InvariantCulture){Delimiter=";"};usingStreamWriterwriter=newStreamWriter("file.csv");usingCsvWritercsv=newCsvWriter(writer,csvWriterConfig);Parallel.ForEach(Enumerable.Range(0,10000), x =>{csv.WriteRecord(x);csv.NextRecord();});
Expected behavior
I excepted the data to be written concurrently to the CSV file.
I read about: #1623 but the last comment states that it is no longer a problem
Additional context
I'm trying to implement a pipeline with the library https://github.com/Open-NET-Libraries/Open.ChannelExtensions starting from an IAsyncEnumerable<T> of data, transforming each item creating a model class, and write those models into multiple CSV files batching by a number. Since I'm dealing with millions of items, I wanted to speed up things and write the data concurrently to the file. To lock asynchronously I use https://github.com/StephenClearyArchive/AsyncEx.Coordination
AsyncLock_evictionLock=newAsyncLock();//Async version of lockCsvHelperFileHandlecsvFile=NewCsvFile();//This is a wrapper of CsvWriterintcount=0;awaitdata//IAsyncEnumerable<T>.ToChannel().PipeAsync(maxConcurrency:100,capacity:1000,transform:CreateModelAsync).ReadAllConcurrentlyAsync(maxConcurrency:100,async model =>{Interlocked.Increment(ref_count);if(Interlocked.CompareExchange(ref_count,0,BatchSize)==0){using(await_evictionLock.LockAsync().ConfigureAwait(false)){csvFile.Dispose();csvFile=NewCsvFile();}}csvFile.Csv.WriteRecord(model);csvFile.Csv.NextRecord();}).ConfigureAwait(false);
The text was updated successfully, but these errors were encountered:
Describe the bug
Calling
WriteRecord
orNextRecord
in a multithreaded scenario causes the error in .NET Framework 4.8:While in
.NET 8
I get another error, not concurrency-related:To Reproduce
I used the same coded in both cases:
Expected behavior
I excepted the data to be written concurrently to the CSV file.
I read about: #1623 but the last comment states that it is no longer a problem
Additional context
I'm trying to implement a pipeline with the library https://github.com/Open-NET-Libraries/Open.ChannelExtensions starting from an
IAsyncEnumerable<T>
of data, transforming each item creating a model class, and write those models into multiple CSV files batching by a number. Since I'm dealing with millions of items, I wanted to speed up things and write the data concurrently to the file. To lock asynchronously I use https://github.com/StephenClearyArchive/AsyncEx.CoordinationThe text was updated successfully, but these errors were encountered: