Take advantage of Stencil buffer in Post Process

Author Avatar
Kanglai Qian 3月 07, 2015

As I posted in 山寨SSSSS before, I cannot find a way to take advantage of stencil buffer in OnRenderImage. This makes the post effect full screen all the time. Other guys have come up with the same question in the formu or AnswerHub. After several days hard work, I finally find a way to use stencil.

Here is an example for fast Bloom:

W/O Stencil:

With Stencil(Ocean Only):

The key idea is keeping the depth buffer (along with stencil) when rendering, with the help of Graphics.SetRenderTarget. First of all, create and set the camera render target. (I also tried depth bit as 16, which makes RenderTexture.SupportsStencil return false, and it still works. I don’t know why…)

RenderTextureFormat format = RenderTextureFormat.Default;
if(SystemInfo.SupportsRenderTextureFormat(RenderTextureFormat.ARGBHalf)){
format = RenderTextureFormat.ARGBHalf;
}
RenderTexture renderTexture = new RenderTexture(Screen.width, Screen.height, 24, format);
renderTexture.Create();
camera.targetTexture = renderTexture;

RenderTexture bloomRT0 = new RenderTexture(renderTexture.width, renderTexture.height, 0, format);
bloomRT0.Create(); //Ensure these two RT have the same size

Instead of downsample in OnRenderImage, we have to Blit in OnPostRender

public void OnPostRender(){
postProcessingMaterial.SetPass(1);

bloomRT0.DiscardContents();
Graphics.SetRenderTarget(bloomRT0);
GL.Clear(true, true, new Color(0,0,0,0)); // clear the full RT
// *KEY POINT*: Draw with the camera's depth buffer
Graphics.SetRenderTarget(bloomRT0.colorBuffer, renderTexture.depthBuffer);
Graphics.Blit(renderTexture, postProcessingMaterial, 1);
}

The shaderlab side is quite simple, and you can find many resources online. Here are two code snippets:

Ocean Shader

Stencil {
Ref 2
Comp Always
Pass replace
}

Post Processing Shader

Stencil{
Ref 2
ReadMask 2
Comp Equal
}

ps. You could try draw to screen directly, without setting an extra renderTexture. Currently I am combining some Post Process together, along with dynamic resolution, which makes this unavoidable.