Allow image scaling on the mjepg stream #138
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| /** | ||
| * Copyright (c) 2015-present, Facebook, Inc. | ||
| * All rights reserved. | ||
| * | ||
| * This source code is licensed under the BSD-style license found in the | ||
| * LICENSE file in the root directory of this source tree. An additional grant | ||
| * of patent rights can be found in the PATENTS file in the same directory. | ||
| */ | ||
|
|
||
| #import <Foundation/Foundation.h> | ||
| #import <CoreGraphics/CoreGraphics.h> | ||
|
|
||
| NS_ASSUME_NONNULL_BEGIN | ||
|
|
||
| /** | ||
| Scales images and compresses it to JPEG using Image I/O | ||
| It allows to enqueue only a single screenshot. If a new one arrives before the currently queued gets discared | ||
| */ | ||
| @interface FBImageIOScaler : NSObject | ||
|
|
||
| /** | ||
| Puts the passed image on the queue and dispatches a scaling operation. If there is already a image on the | ||
| queue it will be replaced with the new one | ||
| @param image The image to scale down | ||
| @param completionHandler called after successfully scaling down an image | ||
| @param scalingFactor the scaling factor in range 0.01..1.0. A value of 1.0 won't perform scaling at all | ||
| @param compressionQuality the compression quality in range 0.0..1.0 (0.0 for max. compression and 1.0 for lossless compression) | ||
| */ | ||
| - (void)submitImage:(NSData *)image scalingFactor:(CGFloat)scalingFactor compressionQuality:(CGFloat)compressionQuality completionHandler:(void (^)(NSData *))completionHandler; | ||
|
|
||
| @end | ||
|
|
||
| NS_ASSUME_NONNULL_END |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| /** | ||
| * Copyright (c) 2015-present, Facebook, Inc. | ||
| * All rights reserved. | ||
| * | ||
| * This source code is licensed under the BSD-style license found in the | ||
| * LICENSE file in the root directory of this source tree. An additional grant | ||
| * of patent rights can be found in the PATENTS file in the same directory. | ||
| */ | ||
|
|
||
| #import "FBImageIOScaler.h" | ||
| #import <ImageIO/ImageIO.h> | ||
| #import <MobileCoreServices/MobileCoreServices.h> | ||
| #import "FBLogger.h" | ||
|
|
||
| static const CGFloat FBMinScalingFactor = 0.01f; | ||
| static const CGFloat FBMaxScalingFactor = 1.0f; | ||
| static const CGFloat FBMinCompressionQuality = 0.0f; | ||
| static const CGFloat FBMaxCompressionQuality = 1.0f; | ||
|
|
||
| @interface FBImageIOScaler () | ||
|
|
||
| @property (nonatomic) NSData *nextImage; | ||
| @property (nonatomic, readonly) NSLock *nextImageLock; | ||
| @property (nonatomic, readonly) dispatch_queue_t scalingQueue; | ||
|
|
||
| @end | ||
|
|
||
| @implementation FBImageIOScaler | ||
|
|
||
| - (id)init | ||
| { | ||
| self = [super init]; | ||
| if (self) { | ||
| _nextImageLock = [[NSLock alloc] init]; | ||
| _scalingQueue = dispatch_queue_create("image.scaling.queue", NULL); | ||
| } | ||
| return self; | ||
| } | ||
|
|
||
| - (void)submitImage:(NSData *)image scalingFactor:(CGFloat)scalingFactor compressionQuality:(CGFloat)compressionQuality completionHandler:(void (^)(NSData *))completionHandler { | ||
| [self.nextImageLock lock]; | ||
| if (self.nextImage != nil) { | ||
| [FBLogger verboseLog:@"Discarding screenshot"]; | ||
| } | ||
| scalingFactor = MAX(FBMinScalingFactor, MIN(FBMaxScalingFactor, scalingFactor)); | ||
| compressionQuality = MAX(FBMinCompressionQuality, MIN(FBMaxCompressionQuality, compressionQuality)); | ||
| self.nextImage = image; | ||
| [self.nextImageLock unlock]; | ||
|
|
||
| dispatch_async(self.scalingQueue, ^{ | ||
| [self.nextImageLock lock]; | ||
| NSData *next = self.nextImage; | ||
| self.nextImage = nil; | ||
| [self.nextImageLock unlock]; | ||
| if (next == nil) { | ||
| return; | ||
| } | ||
| NSData *scaled = [self scaledImageWithImage:next | ||
| scalingFactor:scalingFactor | ||
| compressionQuality:compressionQuality]; | ||
|
There was a problem hiding this comment. do we need to recompress the original image after it has been already compressed by XCTest? Wouldn't this be a waste of resources? There was a problem hiding this comment. Yes, otherwise the size keeps roughly the same. There was a problem hiding this comment. I removed the additional setting for compression quality and now we use the same value for screenshot taking and when we create the jpeg after scaling |
||
| if (scaled == nil) { | ||
| [FBLogger log:@"Could not scale down the image"]; | ||
| return; | ||
| } | ||
| completionHandler(scaled); | ||
| }); | ||
| } | ||
|
|
||
| - (nullable NSData *)scaledImageWithImage:(NSData *)image scalingFactor:(CGFloat)scalingFactor compressionQuality:(CGFloat)compressionQuality { | ||
| CGImageSourceRef imageData = CGImageSourceCreateWithData((CFDataRef)image, nil); | ||
|
|
||
| CGSize size = [FBImageIOScaler imageSizeWithImage:imageData]; | ||
| CGFloat scaledMaxPixelSize = MAX(size.width, size.height) * scalingFactor; | ||
|
|
||
| CFDictionaryRef params = (__bridge CFDictionaryRef)@{ | ||
| (const NSString *)kCGImageSourceCreateThumbnailWithTransform: @(YES), | ||
| (const NSString *)kCGImageSourceCreateThumbnailFromImageIfAbsent: @(YES), | ||
| (const NSString *)kCGImageSourceThumbnailMaxPixelSize: @(scaledMaxPixelSize) | ||
| }; | ||
|
|
||
| CGImageRef scaled = CGImageSourceCreateThumbnailAtIndex(imageData, 0, params); | ||
| if (scaled == nil) { | ||
| [FBLogger log:@"Failed to scale the image"]; | ||
| CFRelease(imageData); | ||
| return nil; | ||
| } | ||
| NSData *jpegData = [self jpegDataWithImage:scaled | ||
| compressionQuality:compressionQuality]; | ||
| CFRelease(scaled); | ||
mykola-mokhnach marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| CFRelease(imageData); | ||
| return jpegData; | ||
| } | ||
|
|
||
| - (nullable NSData *)jpegDataWithImage:(CGImageRef)imageRef compressionQuality:(CGFloat)compressionQuality | ||
| { | ||
| NSMutableData *newImageData = [NSMutableData data]; | ||
| CGImageDestinationRef imageDestination = CGImageDestinationCreateWithData((CFMutableDataRef)newImageData, kUTTypeJPEG, 1, NULL); | ||
|
|
||
| CFDictionaryRef compressionOptions = (__bridge CFDictionaryRef)@{ | ||
| (const NSString *)kCGImageDestinationLossyCompressionQuality: @(compressionQuality) | ||
| }; | ||
|
|
||
| CGImageDestinationAddImage(imageDestination, imageRef, compressionOptions); | ||
| if(!CGImageDestinationFinalize(imageDestination)) { | ||
| [FBLogger log:@"Failed to write the image"]; | ||
| newImageData = nil; | ||
| } | ||
| CFRelease(imageDestination); | ||
| return newImageData; | ||
| } | ||
|
|
||
| + (CGSize)imageSizeWithImage:(CGImageSourceRef)imageSource | ||
| { | ||
| NSDictionary *options = @{ | ||
| (const NSString *)kCGImageSourceShouldCache: @(NO) | ||
| }; | ||
| CFDictionaryRef properties = CGImageSourceCopyPropertiesAtIndex(imageSource, 0, (CFDictionaryRef)options); | ||
|
|
||
| NSNumber *width = [(__bridge NSDictionary *)properties objectForKey:(const NSString *)kCGImagePropertyPixelWidth]; | ||
| NSNumber *height = [(__bridge NSDictionary *)properties objectForKey:(const NSString *)kCGImagePropertyPixelHeight]; | ||
|
|
||
| CGSize size = CGSizeMake([width floatValue], [height floatValue]); | ||
| CFRelease(properties); | ||
| return size; | ||
| } | ||
|
|
||
| @end | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add this setting value in https://github.com/appium/appium/blob/f60b12cae069de3a306b26da622cee0dbbee7d7e/docs/en/advanced-concepts/settings.md#xcuitest after merging this?