#12 Support multi strip encoding. Improve performance and memory usage. #1537

IldarKhayrutdinov · 2021-02-07T09:27:08Z

Prerequisites

I have written a descriptive pull-request title
I have verified that there are no overlapping pull-requests open
I have verified that I am following matches the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
I have provided test coverage for my change (where applicable)

Description

Refactory color encoders and data compressors - it will make it easier to add new color modes (4-bit gray, CMYK, RGB 565/555, YCbCr), compressions (CcittGroup4Fax, Jpeg) and multi page encoding.
Support multi strip encoding.
Improve performance and memory usage of encoders and small improve of decoders.
[fixed by @brianpopow] Report palette lzw bug

Encoing performance improved: 6-69%, memory usage: 8-70%.
Decoder performance stayed ±10%, maybe big tests will show better results.
I don't provided profiling, I think with profiling there is potential for more improvements.

… usage of decoders and encoders. # Conflicts: # tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

IldarKhayrutdinov · 2021-02-07T10:18:49Z

Attention 1

367d8bb#diff-916bcb855ed09d4491b4ccc51fee218af1aa197ec22d499cc4dddc31f6942a20R98-R125
This test I added before my changes, here reproduction incorrect setting color mode from compression. The CcittGroup3Fax and Ccitt1D compressions can use only with biColor mode.
Benchmark code also was corrected (instead of biColor it tested uncompressed rgb):
367d8bb#diff-fdeb1a0d0f29bf3fdecc5db3c8f7110377795fe75843f8858d8e89f5d1e7fdfaL71-L74
The biColor results became a very slowest: da856da
After my changes the test work correctly:
8cca667#diff-916bcb855ed09d4491b4ccc51fee218af1aa197ec22d499cc4dddc31f6942a20R97-R116

IldarKhayrutdinov · 2021-02-07T10:31:30Z

Attention 2

In current ImageSharp:tiff-format branch following tests generate black output images. I don't understand, why they are successful:

ImageSharp/tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

Lines 183 to 201 in c57664d

    
           [Theory] 
        
           [WithFile(Calliphora_PaletteUncompressed, PixelTypes.Rgba32)] 
        
           public void TiffEncoder_EncodeColorPalette_WithLzwCompression_Works<TPixel>(TestImageProvider<TPixel> provider) 
        
               where TPixel : unmanaged, IPixel<TPixel> 
        
           { 
        
               var encoder = new TiffEncoder { Mode = TiffEncodingMode.ColorPalette, Compression = TiffEncoderCompression.Lzw }; 
        
               this.TiffEncoderPaletteTest(provider, encoder); 
        
           } 
        
           [Theory] 
        
           [WithFile(Calliphora_PaletteUncompressed, PixelTypes.Rgba32)] 
        
           public void TiffEncoder_EncodeColorPalette_WithLzwCompressionAndPredictor_Works<TPixel>(TestImageProvider<TPixel> provider) 
        
               where TPixel : unmanaged, IPixel<TPixel> 
        
           { 
        
               var encoder = new TiffEncoder { Mode = TiffEncodingMode.ColorPalette, Compression = TiffEncoderCompression.Lzw, UseHorizontalPredictor = true }; 
        
               this.TiffEncoderPaletteTest(provider, encoder); 
        
           }

Also I tested this bug in revision befor merging #1527, the bug was there too.

This bug I found during multistrip tests. Therefore I add workaround for this tests and add bug reprodusion test:
5fc3d37#diff-916bcb855ed09d4491b4ccc51fee218af1aa197ec22d499cc4dddc31f6942a20R210-R237

brianpopow · 2021-02-07T14:12:20Z

Attention 2

In current ImageSharp:tiff-format branch following tests generate black output images. I don't understand, why they are successful:

The test do not fail because with the palette image tests, we do not compare the result against the input image. There is a quantization step involved to create the palette, which means the encoded image will always be slightly different from the original.
Instead we load the encoded image with a reference decoder and compare against that. Not sure how we can improve that.

I tried to figure out the issue, but so far i cannot see any obvious bug. tiffinfo shows no error with the encoded image:

TIFF Directory at offset 0x1c1e (7198)
  Image Width: 804 Image Length: 1198
  Resolution: 96, 96 pixels/inch
  Bits/Sample: 8
  Compression Scheme: LZW
  Photometric Interpretation: palette color (RGB from colormap)
  Samples/Pixel: 1
  Rows/Strip: 1198
  Planar Configuration: single image plane
  Color Map: (present)
  DocumentName: .\Calliphora_palette_uncompressed.tiff
  ImageDescription: File source: http://commons.wikimedia.org/wiki/File:Calliphora_sp_Portrait.jpg
  Software: ImageSharp

IldarKhayrutdinov · 2021-02-07T14:36:29Z

@brianpopow there are my outputs EncodeColorPalette_WithLzw.zip
from \ImageSharp\tests\Images\ActualOutput\TiffEncoderTests\ path.

There may be a problem on my computer, please check your output?

brianpopow · 2021-02-07T14:47:06Z

@brianpopow there are my outputs EncodeColorPalette_WithLzw.zip
from \ImageSharp\tests\Images\ActualOutput\TiffEncoderTests\ path.

There may be a problem on my computer, please check your output?

@IldarKhayrutdinov i can reproduce the isssue, i do see those black images, too. I just dont know yet why this is happening.

IldarKhayrutdinov · 2021-02-07T15:00:30Z

@brianpopow there are my outputs EncodeColorPalette_WithLzw.zip
from \ImageSharp\tests\Images\ActualOutput\TiffEncoderTests\ path.
There may be a problem on my computer, please check your output?

@IldarKhayrutdinov i can reproduce the isssue, i do see those black images, too. I just dont know yet why this is happening.

ok, I want to note that this problem was before this PR

IldarKhayrutdinov · 2021-02-07T15:53:56Z

Encoder performance report

Spoiler

Method	Runtime	Compression	Mean	Ratio	Allocated	Improve performance	Improve memory allocation
'System.Drawing Tiff'	.NET 4.7.2	None	6.520 ms	1.00**	11570062 B
'ImageSharp Tiff'	.NET 4.7.2	None	5.698 ms	0.87	9919288 B
PR 'ImageSharp Tiff'	.NET 4.7.2	None	4.844 ms	0.73	7445922 B	6%	25%
'System.Drawing Tiff'	.NET Core 2.1	None	6.851 ms	1.00	11562768 B
'ImageSharp Tiff'	.NET Core 2.1	None	4.294 ms	0.63	9918144 B
PR 'ImageSharp Tiff'	.NET Core 2.1	None	3.189 ms	0.46	7444718 B	27%	25%
'System.Drawing Tiff'	.NET Core 3.1	None	5.835 ms	1.00	8672224 B
'ImageSharp Tiff'	.NET Core 3.1	None	5.167 ms	0.89	9918112 B
PR 'ImageSharp Tiff'	.NET Core 3.1	None	3.342 ms	0.57	7444631 B	36%	25%
'System.Drawing Tiff'	.NET 4.7.2	Deflate	NA	?	-
'ImageSharp Tiff'	.NET 4.7.2	Deflate	125.909 ms	?	11167960 B
PR 'ImageSharp Tiff'	.NET 4.7.2	Deflate	87.815 ms	?	6617521 B	30%	40%
'System.Drawing Tiff'	.NET Core 2.1	Deflate	NA	?	-
'ImageSharp Tiff'	.NET Core 2.1	Deflate	125.041 ms	?	11164792 B
PR 'ImageSharp Tiff'	.NET Core 2.1	Deflate	84.005 ms	?	6605507 B	33%	40%
'System.Drawing Tiff'	.NET Core 3.1	Deflate	NA	?	-
'ImageSharp Tiff'	.NET Core 3.1	Deflate	125.139 ms	?	11168428 B
PR 'ImageSharp Tiff'	.NET Core 3.1	Deflate	81.102 ms	?	6604792 B	35%	40%
'System.Drawing Tiff'	.NET 4.7.2	Lzw	49.024 ms	1.00	10673371 B
'ImageSharp Tiff'	.NET 4.7.2	Lzw	411.728 ms	8.41	23265464 B
PR 'ImageSharp Tiff'	.NET 4.7.2	Lzw	125.569 ms	2.66	8423760 B	69%	64%
'System.Drawing Tiff'	.NET Core 2.1	Lzw	47.288 ms	1.00	10668688 B
'ImageSharp Tiff'	.NET Core 2.1	Lzw	201.643 ms	4.26	27451168 B
PR 'ImageSharp Tiff'	.NET Core 2.1	Lzw	96.217 ms	2.03	8422488 B	52%	70%
'System.Drawing Tiff'	.NET Core 3.1	Lzw	46.526 ms	1.00	8001741 B
'ImageSharp Tiff'	.NET Core 3.1	Lzw	170.276 ms	3.66	27451445 B
PR 'ImageSharp Tiff'	.NET Core 3.1	Lzw	93.635 ms	2.02	8422504 B	44%	70%
'System.Drawing Tiff'	.NET 4.7.2	PackBits	NA	?	-
'ImageSharp Tiff'	.NET 4.7.2	PackBits	28.948 ms	?	9943858 B
PR 'ImageSharp Tiff'	.NET 4.7.2	PackBits	27.449 ms	?	7453052 B	5%	25%
'System.Drawing Tiff'	.NET Core 2.1	PackBits	NA	?	-
'ImageSharp Tiff'	.NET Core 2.1	PackBits	22.611 ms	?	9942792 B
PR 'ImageSharp Tiff'	.NET Core 2.1	PackBits	19.935 ms	?	7451912 B	12%	25%
'System.Drawing Tiff'	.NET Core 3.1	PackBits	NA	?	-
'ImageSharp Tiff'	.NET Core 3.1	PackBits	23.465 ms	?	9942772 B
PR 'ImageSharp Tiff'	.NET Core 3.1	PackBits	19.664 ms	?	7451974 B	16%	25%
'System.Drawing Tiff'	.NET 4.7.2	CcittGroup3Fax	43.618 ms	1.00	1169683 B
'ImageSharp Tiff'	.NET 4.7.2	CcittGroup3Fax	191.602 ms	4.39	24829048 B
PR 'ImageSharp Tiff'	.NET 4.7.2	CcittGroup3Fax	191.413 ms	4.42	22714336 B	-1%	8%
'System.Drawing Tiff'	.NET Core 2.1	CcittGroup3Fax	43.258 ms	1.00	1169200 B
'ImageSharp Tiff'	.NET Core 2.1	CcittGroup3Fax	177.930 ms	4.11	24772997 B
PR 'ImageSharp Tiff'	.NET Core 2.1	CcittGroup3Fax	180.059 ms	4.13	22658509 B	-	8%
'System.Drawing Tiff'	.NET Core 3.1	CcittGroup3Fax	43.330 ms	1.00	850189 B
'ImageSharp Tiff'	.NET Core 3.1	CcittGroup3Fax	168.846 ms	3.90	24774571 B
PR 'ImageSharp Tiff'	.NET Core 3.1	CcittGroup3Fax	171.370 ms	3.94	22658261 B	-1%	8%
'System.Drawing Tiff'	.NET 4.7.2	ModifiedHuffman	17.106 ms	1.00	11561706 B
'ImageSharp Tiff'	.NET 4.7.2	ModifiedHuffman	192.530 ms	11.27	24826163 B
PR 'ImageSharp Tiff'	.NET 4.7.2	ModifiedHuffman	191.066 ms	11.18	22710384 B	<1%	8%
'System.Drawing Tiff'	.NET Core 2.1	ModifiedHuffman	16.988 ms	1.00	11555088 B
'ImageSharp Tiff'	.NET Core 2.1	ModifiedHuffman	180.265 ms	10.61	24769453 B
PR 'ImageSharp Tiff'	.NET Core 2.1	ModifiedHuffman	177.379 ms	10.41	22656395 B	1%	8%
'System.Drawing Tiff'	.NET Core 3.1	ModifiedHuffman	15.989 ms	1.00	8666467 B
'ImageSharp Tiff'	.NET Core 3.1	ModifiedHuffman	181.295 ms	11.34	24770275 B
PR 'ImageSharp Tiff'	.NET Core 3.1	ModifiedHuffman	167.231 ms	10.49	22659275 B	7%	8%

IldarKhayrutdinov · 2021-02-07T16:06:28Z

There were a lot of changes, don't hesitate to comments 😉

src/ImageSharp/Formats/Tiff/TiffEncoderPixelStorageMethod.cs

src/ImageSharp/Formats/Tiff/TiffEncoderCore.cs

src/ImageSharp/Formats/Tiff/Writers/TiffBiColorWriter.cs

IldarKhayrutdinov · 2021-02-07T16:43:07Z

src/ImageSharp/Formats/Tiff/Writers/TiffPaletteWriter.cs

        {
+            using IQuantizer<TPixel> frameQuantizer = quantizer.CreatePixelSpecificQuantizer<TPixel>(this.Configuration);
+            this.quantized = frameQuantizer.BuildPaletteAndQuantizeFrame(image, image.Bounds());


Palette encoder also use hard operations.
I didn't change this code, stayed as is.

This should utilize whatever bit depth property we store.

@JimBobSquarePants Sorry, I don't understand, what did you mean?

I'm seeing a passed IQuantizer instance but also hardcoded ColorsPerChannel values which indicate 256 colors each time. I would assume that there is some sort of BitDepth property that is captured during decoding and can be optionally set to dictate the encoding bit depth. There should be consistency in the palette size and desired bit depth.

Yeah, it's required to bit depth settings, but at the moment support only 8 bit palette and grayscale.
I would like add bit depth later, in future PR.

IldarKhayrutdinov · 2021-02-07T16:45:59Z

src/ImageSharp/Formats/Tiff/TiffEncoderCore.cs

+            {
+                default:
+                case TiffEncoderPixelStorageMethod.Auto:
+                case TiffEncoderPixelStorageMethod.MultiStrip:


default use multi strip method

IldarKhayrutdinov · 2021-02-07T16:51:48Z

src/ImageSharp/Formats/Tiff/ITiffEncoderOptions.cs

+        /// <summary>
+        /// Gets the maximum size of strip (bytes).
+        /// </summary>
+        int MaxStripBytes { get; }


this is a limit of one strip length (the field also can be used for tiles)

this field also can be renamed 🙂
example MaxStripLength

i think MaxStripBytes is an ok name, its clear what it does.

Is that interface public? I’m curious as to what the property means as a configuration value to the average user

The interface is internal, but TiffEncoder class that inherit this interface is public.
I looked StripByteCounts on different tiff files and watched that they have different lengths, not only 8Kb, in some cases it reaches 1Mb. So I figured that the user might want to set the strip size.

Pardon my ignorance but what do the different strip count length mean in terms of encoding?

In simple terms this is size of encoder (and decoder when read) buffer, the image is divided into strips and encoding one by one, so there is no need to allocate large buffers for full image size. It seems that there is no single agreement, there is only a recommendation about 8Kb (also I read about 64Kb), I think most users will be satisfied with default size. But maybe someone wants to adjust the strip size himself. Maybe @brianpopow will give more complete information?

frankly, for modern computers, I think it won't be a problem using 1Mb buffers, but 8Kb is safer

Thanks... I guess we can determine a sensible default by benchmarking.

src/ImageSharp/Formats/Tiff/Compression/Compressors/DeflateCompressor.cs

brianpopow · 2021-02-12T12:21:08Z

@IldarKhayrutdinov there seems to be an issue now with the packedbits compression. While the thumbnail of the tiffs look fine, if you open the image it looks broken. See for example in the Actual output folder of the tests: TiffEncoder_EncodeColorPalette_WithPackBitsCompression_Works_Rgba32_Calliphora_palette_uncompressed.tiff

Maybe we should add a CompareToReference output step in the tests to cover those cases.

…rrect tests.

IldarKhayrutdinov · 2021-02-13T10:41:20Z

@IldarKhayrutdinov there seems to be an issue now with the packedbits compression. While the thumbnail of the tiffs look fine, if you open the image it looks broken. See for example in the Actual output folder of the tests: TiffEncoder_EncodeColorPalette_WithPackBitsCompression_Works_Rgba32_Calliphora_palette_uncompressed.tiff

Maybe we should add a CompareToReference output step in the tests to cover those cases.

@brianpopow here is my output, it seems it looks normal, maybe different results on different computers?
TiffEncoder_EncodeColorPalette_WithPackBitsCompression_Works_Rgba32_Calliphora_palette_uncompressed.zip

I think adding CompareToReference is a good idea.

brianpopow · 2021-02-13T15:34:55Z

@IldarKhayrutdinov there seems to be an issue now with the packedbits compression. While the thumbnail of the tiffs look fine, if you open the image it looks broken. See for example in the Actual output folder of the tests: TiffEncoder_EncodeColorPalette_WithPackBitsCompression_Works_Rgba32_Calliphora_palette_uncompressed.tiff
Maybe we should add a CompareToReference output step in the tests to cover those cases.

@brianpopow here is my output, it seems it looks normal, maybe different results on different computers?
TiffEncoder_EncodeColorPalette_WithPackBitsCompression_Works_Rgba32_Calliphora_palette_uncompressed.zip

I think adding CompareToReference is a good idea.

I doubt that the output can be different on different computers. We do not use any hardware intrinsics when decoding tiffs.

After digging a bit deeper and that you mentioned it does not look corrupted on your pc, it seems an issue with IrfanView, gimp and XnView. On the other hand viewing it wit paint works. I definitely think there is a regression introduced with this PR and PackedBits compression, as i dont see this issue with the current tiff branch.

I will try to improve on the tests so we will spot those issues.

Here is an example output:

IldarKhayrutdinov · 2021-02-13T16:17:13Z

I doubt that the output can be different on different computers. We do not use any hardware intrinsics when decoding tiffs.

yes, it confused me too...

now I also saw through IrfanView (earlier I looked through the standard windows viewer and MS Paint)

IldarKhayrutdinov · 2021-02-14T14:14:12Z

@brianpopow I investigated the lzw issue and finally understood, cause of decoder affecting for encoder test.
The encoder test method:

ImageSharp/tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

Lines 222 to 226 in ed6078e

    
           image.Save(memStream, encoder); 
        
           memStream.Position = 0; 
        
           using var encodedImage = (Image<TPixel>)Image.Load(this.configuration, memStream); 
        
           var encodedImagePath = provider.Utility.SaveTestOutputFile(encodedImage, "tiff", encoder);

line using var encodedImage = (Image<TPixel>)Image.Load(this.configuration, memStream);

here output image reload and resave, so in output directory tests\Images\ActualOutput\TiffEncoderTests placed resaved image.

brianpopow · 2021-02-14T14:20:26Z

@brianpopow I investigated the lzw issue and finally understood, cause of decoder affecting for encoder test.
The encoder test method:

ImageSharp/tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

Lines 222 to 226 in ed6078e

image.Save(memStream, encoder);

memStream.Position = 0;

using var encodedImage = (Image<TPixel>)Image.Load(this.configuration, memStream);

var encodedImagePath = provider.Utility.SaveTestOutputFile(encodedImage, "tiff", encoder);

line using var encodedImage = (Image<TPixel>)Image.Load(this.configuration, memStream);

here output image reload and resave, so in output directory tests\Images\ActualOutput\TiffEncoderTests placed resaved image.

yeah that was an unlucky workaround for the fact that the quantization was changing the output image slightly. I have changed that part now and use a TolerantComparer for the color palette tests: a8f75b0. This should fail now if the output has too much different from the input.

IldarKhayrutdinov · 2021-02-14T14:21:49Z

yeah that was an unlucky workaround for the fact that the quantization was changing the output image slightly. I have changed that part now and use a TolerantComparer for the color palette tests: a8f75b0. This should fail now if the output has too much different from the input.

👍 after, I'll sync with tiff-format branch

I researching loading issue with IrfanView

JimBobSquarePants · 2021-02-14T16:22:33Z

using var encodedImage = (Image<TPixel>)Image.Load(this.configuration, memStream);

Why the cast here and not using the generic version directly? That cast will only work if TPixel is Rgba32

IldarKhayrutdinov · 2021-02-14T17:52:42Z

Why the cast here and not using the generic version directly? That cast will only work if TPixel is Rgba32

@JimBobSquarePants now this method has removed, now use TestTiffEncoderCore method:

ImageSharp/tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

Line 344 in 0b8eac5

    
           image.VerifyEncoder(provider, "tiff", bitsPerPixel, encoder, useExactComparer ? ImageComparer.Exact : ImageComparer.Tolerant(compareTolerance), referenceDecoder: ReferenceDecoder);

IldarKhayrutdinov · 2021-02-14T18:10:57Z

@brianpopow I fixed PackBits bug, please check, is everything fixed?

brianpopow · 2021-02-14T18:17:00Z

@brianpopow I fixed PackBits bug, please check, is everything fixed?

@IldarKhayrutdinov great job in finding this issue! I will try to get this reviewed tomorrow.

brianpopow

Looks good, @IldarKhayrutdinov thank you once again for your help!

#12 Support multi strip encoding. Improve performance and memory usage.

IldarKhayrutdinov added 4 commits February 7, 2021 11:49

Report 1-bit mode bug. Enable test images. Change test files.

367d8bb

Add benchmarks results

da856da

Support multi strip encoding for tiff. Improve performance and memory…

8cca667

… usage of decoders and encoders. # Conflicts: # tests/ImageSharp.Tests/Formats/Tiff/TiffEncoderTests.cs

Report palette lzw bug

5fc3d37

Update benchmark results

f1e1e21

IldarKhayrutdinov force-pushed the tiff-format branch from bbbb465 to f1e1e21 Compare February 7, 2021 14:58

IldarKhayrutdinov marked this pull request as ready for review February 7, 2021 16:00

IldarKhayrutdinov mentioned this pull request Feb 7, 2021

Tiff format support #1456

Closed

17 tasks

IldarKhayrutdinov commented Feb 7, 2021

View reviewed changes

IldarKhayrutdinov added 3 commits February 7, 2021 22:02

DebugGuard fixes

93a6349

Remove excess benchmark reports

e74fa78

Update readme

bc75ca3

brianpopow reviewed Feb 10, 2021

View reviewed changes

src/ImageSharp/Formats/Tiff/Compression/Compressors/DeflateCompressor.cs Outdated Show resolved Hide resolved

Change BinaryDither to FloydSteinberg

4895521

Remove TiffEncoderPixelStorageMethod, add CRC writing for deflate. Co…

34a4f77

…rrect tests.

IldarKhayrutdinov added 2 commits February 14, 2021 17:38

Merge remote-tracking branch 'upstream/tiff-format' into tiff-format

be08ae1

Rename tests

0b8eac5

brianpopow added the formats:tiff label Feb 14, 2021

IldarKhayrutdinov added 2 commits February 14, 2021 19:22

Correct test

8f77d2f

PackBits bug fix

e8c04dd

Remove excess clearing

35cb6fd

IldarKhayrutdinov requested review from JimBobSquarePants and brianpopow February 14, 2021 18:11

Clarify some DebugGuard messages and a little cleanup

dfc7292

brianpopow approved these changes Feb 15, 2021

View reviewed changes

brianpopow merged commit 6774c57 into SixLabors:tiff-format Feb 15, 2021

IldarKhayrutdinov deleted the tiff-format branch February 15, 2021 15:43

JimBobSquarePants pushed a commit that referenced this pull request Feb 17, 2021

Merge pull request #1537 from IldarKhayrutdinov/tiff-format

a04e598

#12 Support multi strip encoding. Improve performance and memory usage.

JimBobSquarePants pushed a commit that referenced this pull request Feb 17, 2021

Merge pull request #1537 from IldarKhayrutdinov/tiff-format

2ec4787

#12 Support multi strip encoding. Improve performance and memory usage.

#12 Support multi strip encoding. Improve performance and memory usage. #1537

#12 Support multi strip encoding. Improve performance and memory usage. #1537

Conversation

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

Prerequisites

Description

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

brianpopow commented Feb 7, 2021

IldarKhayrutdinov commented Feb 7, 2021

brianpopow commented Feb 7, 2021

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

IldarKhayrutdinov commented Feb 7, 2021 • edited Loading

IldarKhayrutdinov Feb 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IldarKhayrutdinov Feb 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IldarKhayrutdinov Feb 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

IldarKhayrutdinov Feb 12, 2021 • edited Loading

Choose a reason for hiding this comment

IldarKhayrutdinov Feb 12, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianpopow commented Feb 12, 2021

IldarKhayrutdinov commented Feb 13, 2021 • edited Loading

brianpopow commented Feb 13, 2021

IldarKhayrutdinov commented Feb 13, 2021

IldarKhayrutdinov commented Feb 14, 2021

brianpopow commented Feb 14, 2021

IldarKhayrutdinov commented Feb 14, 2021 • edited Loading

JimBobSquarePants commented Feb 14, 2021

IldarKhayrutdinov commented Feb 14, 2021 • edited Loading

IldarKhayrutdinov commented Feb 14, 2021

brianpopow commented Feb 14, 2021

brianpopow left a comment

Choose a reason for hiding this comment

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov Feb 7, 2021 •

edited

Loading

IldarKhayrutdinov Feb 11, 2021 •

edited

Loading

IldarKhayrutdinov Feb 12, 2021 •

edited

Loading

IldarKhayrutdinov Feb 12, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 13, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 14, 2021 •

edited

Loading

IldarKhayrutdinov commented Feb 14, 2021 •

edited

Loading