screen_capture_lite: Screen capturing is broken on OS X devices
Hello!
Current implementation of CGFrameProcessor::ProcessFrame() seems to be unreliable and, sometimes, completely broken.
Apple CG’s documentation is far from complete, but here are some points i discovered while trying to make screenshot myself:
- Image width may be actually smaller, than pixels per row in image data. (this was my main problem)
- Image bitmap data might be stored in different formats across different devices.
How to deal with that?
Here is how I deal with these problems:
// First, capture the image
CGImageRef const capturedImage = /* any way of capturing image, be it CGWindowListCreateImage or CGDisplayCreateImage */;
// Second, get bitmap and alpha data info
CGBitmapInfo const bitmapInfo = CGImageGetBitmapInfo(capturedImage);
CGImageAlphaInfo const alphaInfo = CGImageGetAlphaInfo(capturedImage);
In order to do screenshots on any platform, one must be ready to handle any combination of bitmap and alpha structures. Now I will only look at 32-bit Little-Endian bitmap format with Alpha being the first byte. Note that alpha is unused in screenshot capturing, so we should not care about its representation.
// Ensure 32-bit LE
assert(bitmapInfo & kCGBitmapByteOrder32Little);
// Ensure alpha is at the first byte and has any format (we'll just skip it while reading)
assert(
alphaInfo == kCGImageAlphaFirst ||
alphaInfo == kCGImageAlphaNoneSkipFirst ||
alphaInfo == kCGImageAlphaPremultipliedFirst);
Many implementations add padding at the end of each pixel row. In my case (1366x768 screen), it was 10 completely transparent pixels.
// Get real image width and height, this has NOTHING to do with size of image data provided
std::size_t const width = CGImageGetWidth(capturedImage);
std::size_t const height = CGImageGetHeight(capturedImage);
// This is the actual row size. Many implementations add some padding at the end of each row
std::size_t const bytesPerRow = CGImageGetBytesPerRow(capturedImage);
std::size_t const bytesPerPixel = CGImageGetBitsPerPixel(capturedImage) / 8u;
std::size_t const pixelsPerRow = bytesPerRow / bytesPerPixel;
Now let’s get the data provider and copy all the data from it:
GDataProviderRef const dataProvider = CGImageGetDataProvider(capturedImage);
if (!dataProvider) {
// Handle this. I think that this can only happen if the passed CGImageRef was NULL
}
// As this will copy the data, it needs to be freed separately from the captured image.
CFDataRef const rawPixelData = CGDataProviderCopyData(dataProvider);
Then the data can be accessed as you like. Make sure to take into account bitmap and alpha info and perform all needed conversions for library’s internal format
// Cast the byte pointer to whatever you like
uint32_t const * const dataPointer = reinterpret_cast<uint32_t const *>(CFDataGetBytePtr(rawPixelData));
// And access it like this:
for (unsigned row = 0; row < height; ++row) {
for (unsigned column = 0; column < width; ++column) {
// Note that pixelsPerRow includes padding pixels
/* whatever */ = dataPointer[row * pixelsPerRow + column];
}
}
Also note that src/ios/CGFrameProcessor.cpp is a mix of 4-space and tab-character indentation. This is an important issue, as tabulation character may be displayed as four, two or even eight spaces.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 20 (14 by maintainers)
Updated the Library, example and decided on a pixel format as well. onImage added as a callback to change the image Also, the common pixel format on Mac,Windows and Linux is BGRA, so i have decided to use that format.
Let me know if this works for you…