Looking for some CIKernelROICallback confirmation

This is my first time using CIKernelROICallback, and while it solved the problem of only a portain of my image being processed; I am now getting what I beleive are to be GPU crashes; whereby the resulting image is corrupted and my application has to be force quit (after rendering it's still using 100% of the CPU).


I have two questions.

1. Can someone look at the code below and confirm that it looks correct please.

// called when setting up for fragment program and also calls fragment program
- (CIImage *)outputImage
{
    // --- Apple's example used a float; but I tried it as a double.
    double radius = [blurRadius doubleValue];
    CISampler *src = [CISampler samplerWithImage:inputImage];
    CISampler *envMap = [CISampler samplerWithImage:map];
    
    CGRect mapExtent = [map extent]; // --- Grab th amp extent.
    
    // --- This is the callback that we cannot do in Xojo.
    CIKernelROICallback callback = ^(int index, CGRect rect) {
        if ( index == 0 ) {
            return CGRectInset( rect, -radius, -radius );
        } else {
            return mapExtent;
        }       
        return rect;
    };
    
    // --- This following line was missing from Apple's example, it  allows the kernel
    //     to process with a radius of 200, but still crashes at 500 or 1000.
    CGRect dod = CGRectInset( [inputImage extent], -radius, -radius );
    
    return [_OCIGrouperFilterKernel applyWithExtent:dod
                                        roiCallback:callback
                                          arguments:@[ src,
                                                       envMap,
                                                       sampleCount,
                                                       spatialWC,
                                                       rangeWC,
                                                       clampScale]];
}


2. Before the image is passed to this filter; I use [CIImage imageByClampingToExtent:] and when the image is returned from the filter I call [CIImage imageByCroppingToRect:] ( using the extent before processing). Should the DOD be the inset extent of the image before it was clampped?


i.e.

CGRect dod = CGRectInset( unclampedExtent, -radius, -radius );


Thanks for any help you can give; this section has been particularly painful and frustrating.

Answered by rowlands in 349412022

On a whim, I tried something and it fixed it (I've only tested with 3 images so far, but these 3 image made it crash). The key is that the extent *should* be the extent *before* the image has been clamped. I moved the clamping and cropping into the filter.


// called when setting up for fragment program and also calls fragment program
- (CIImage *)outputImage
{
    CGRect dod = [inputImage extent];
    CGRect mapExtent = [map extent];
    double radius = [blurRadius doubleValue];
    CISampler *src = [CISampler samplerWithImage:[inputImage imageByClampingToExtent]];
    CISampler *envMap = [CISampler samplerWithImage:map];
    
    CIKernelROICallback callback = ^(int index, CGRect rect) {
        if ( index == 0 ) { return CGRectInset( rect, -radius, -radius );
        } else { return mapExtent; }
    };
    
    CIImage *rValue = [_OCIGrouperFilterKernel applyWithExtent:dod
                                                   roiCallback:callback
                                                     arguments:@[ src,
                                                                  envMap,
                                                                  sampleCount,
                                                                  spatialWC,
                                                                  rangeWC,
                                                                  clampScale]];
    return [rValue imageByCroppingToRect:dod];
}

What is

blurRadius
used for in the filter? It's not passed as parameter directly...

I assume it somehow influences the size of the sampled region. In this case the ROI seem to be correct (you're going to need to sampel pixels outside of the original bounds). This also means that

imageByClampingToExtent
sounds like a good idea (to avoid "bleeding" of transparent pixels inside from the image borders).


The DOD describes the region for which your filter produces meaningful, non-transperent results. So unless your filter produces an image that is actually larger then the original input, you could just set the DOD to

[inputImage extend]
.


Concerning the crash: I assume it's due to the very large radius you are using. If I assumed correctly about its meaning, then your filter would sample a region of

(2*radius+1)^2
pixels for each input pixel. For radii of 200 and greater this is huge! I guess the GPU just can't handle that.

What is

blurRadius
used for in the filter? It's not passed as parameter directly...

Correct, this kernel gets passed the "map" sampler, which contains a list of PDS points that it's to read from. I only added the blurRadius property to the filter so it could be used in the callback.


I assume it somehow influences the size of the sampled region. In this case the ROI seem to be correct (you're going to need to sampel pixels outside of the original bounds).

Excellent, thanks for confirming this.


The DOD describes the region for which your filter produces meaningful, non-transperent results. So unless your filter produces an image that is actually larger then the original input, you could just set the DOD to

[inputImage extend]
.

Okay, that's good. So I didn't really understand what it was for, should I pass in the [CIImage extent] before I clamp it?


Concerning the crash: I assume it's due to the very large radius you are using. If I assumed correctly about its meaning, then your filter would sample a region of

(2*radius+1)^2
pixels for each input pixel. For radii of 200 and greater this is huge! I guess the GPU just can't handle that.

Yeah, that's kind of what I assumed is happening. It works (fine I assume) for a screen sized image, but the full resolution *can* crash, although it doesn't always. CPU rendering works, but it can take minutes as opposed to seconds. What's troubling is when it does crash, my application can't be quit, it has to be force quit. So I'm thinking to use a helper application (and shared memory) to do the full size render, so at least if it crashes, I can force quit the helper and alert the user that they need to use CPU rendering (or auto switch to CPU rendering and try again).


Thanks Frank, I really appreciate your help.

Accepted Answer

On a whim, I tried something and it fixed it (I've only tested with 3 images so far, but these 3 image made it crash). The key is that the extent *should* be the extent *before* the image has been clamped. I moved the clamping and cropping into the filter.


// called when setting up for fragment program and also calls fragment program
- (CIImage *)outputImage
{
    CGRect dod = [inputImage extent];
    CGRect mapExtent = [map extent];
    double radius = [blurRadius doubleValue];
    CISampler *src = [CISampler samplerWithImage:[inputImage imageByClampingToExtent]];
    CISampler *envMap = [CISampler samplerWithImage:map];
    
    CIKernelROICallback callback = ^(int index, CGRect rect) {
        if ( index == 0 ) { return CGRectInset( rect, -radius, -radius );
        } else { return mapExtent; }
    };
    
    CIImage *rValue = [_OCIGrouperFilterKernel applyWithExtent:dod
                                                   roiCallback:callback
                                                     arguments:@[ src,
                                                                  envMap,
                                                                  sampleCount,
                                                                  spatialWC,
                                                                  rangeWC,
                                                                  clampScale]];
    return [rValue imageByCroppingToRect:dod];
}

Oh, right, otherwise the image would have a (virtually) infinite extent.


By the way, you can achieve the same result by setting the

kCISamplerWrapMode
option (when initializing the
CISampler
) to
kCISamplerWrapClamp
. Then you don't need to apply
imageByClampingToExtent
.

Oh cool!


I think this is all starting to make some sense now...

Looking for some CIKernelROICallback confirmation
 
 
Q