iDevGames Forums
Masking without a stencil buffer? - Printable Version

+- iDevGames Forums (http://www.idevgames.com/forums)
+-- Forum: Development Zone (/forum-3.html)
+--- Forum: Graphics & Audio Programming (/forum-9.html)
+--- Thread: Masking without a stencil buffer? (/thread-2147.html)



Masking without a stencil buffer? - Bachus - Nov 14, 2008 01:23 PM

First, an image:

[Image: mask_problem.jpg]

On the left is a masked off area of the screen. On the right is random colors drawn where the mask is. Robocop represents some random complex 3D background.

I need to draw a large image (the color gradient) over only part of the screen (the masked off area). The large image doesn't change, however it does scroll around relative to the mask (the mask doesn't move or change). So I can't just slap an alpha map on the image and call it a day. Ordinarily this would be easy to accomplish with the stencil buffer, however this is for the iPhone that has no stencil buffer.

Any ideas?

EDIT: Here's another image to better explain what I'm looking for. Plus I'm bored:

[Image: mask_problem.gif]


Masking without a stencil buffer? - mattz - Nov 14, 2008 01:47 PM

I can't tell if the 3D background (henceforth Robocop) changes or not. If not: then render your rainbow (or whatever first). Put the mask as an alpha channel in the robocop image and draw it OVER the rainbow.

If robocop needs to change, first draw him. Then set up automatic texture coordinate generation so that it is linear in screen space. Then render your mask as an opaque triangle mesh with the rainbow texture.

Does that work?


Masking without a stencil buffer? - Bachus - Nov 14, 2008 01:56 PM

mattz Wrote:I can't tell if the 3D background (henceforth Robocop) changes or not. If not: then render your rainbow (or whatever first). Put the mask as an alpha channel in the robocop image and draw it OVER the rainbow.

If robocop needs to change, first draw him. Then set up automatic texture coordinate generation so that it is linear in screen space. Then render your mask as an opaque triangle mesh with the rainbow texture.

Does that work?

Yea, Robocop just represents a 3D background that does change. So it has to be drawn first.

The problem with with the triangle mesh is that right now the mask is only in image format and is fairly complex (the star is a simplification). It would take a lot of triangles to approximate it properly.


Masking without a stencil buffer? - mattz - Nov 14, 2008 02:27 PM

Does the iPhone do fragment shaders and multitexturing? If so, it's trivial to write a fragment program that does what you want.

Otherwise, maybe you can use the depth buffer as a stencil buffer. Recipe:

- draw robocop
- clear the depth buffer
- enable alpha testing
- draw a single screen-aligned quad using the mask image as an alpha texture. this discards all fragments not in the mask
- disable alpha testing
- change the depth test to GL_EQUAL
- draw the same screen-aligned quad using the rainbow as a color texture
- profit!


Masking without a stencil buffer? - OneSadCookie - Nov 14, 2008 03:09 PM

You've got texture_env_combine and 2 stages, so render the 3D image first, then disable depth test, draw a full-screen quad with combine set up to cut out the star from the rainbow image.


Masking without a stencil buffer? - Bachus - Nov 14, 2008 03:10 PM

mattz Wrote:Does the iPhone do fragment shaders and multitexturing? If so, it's trivial to write a fragment program that does what you want.

Alas, but no. Sad EDIT: Well multitexturing: yes, but shaders: no.

Quote:Otherwise, maybe you can use the depth buffer as a stencil buffer. Recipe:

- draw robocop
- clear the depth buffer
- enable alpha testing
- draw a single screen-aligned quad using the mask image as an alpha texture. this discards all fragments not in the mask
- disable alpha testing
- change the depth test to GL_EQUAL
- draw the same screen-aligned quad using the rainbow as a color texture
- profit!

That would probably work. I shall give it a try after I get back from Bond, James Bond.


Masking without a stencil buffer? - Frogblast - Nov 14, 2008 04:07 PM

As OneSadCookie said, this can be achieved by creating an alpha texture from your mask, and using some texture environment features to combine that with your mask. As a bonus, this will also get you soft edges if you want them, and you also don't have to generate geometry to populate the stencil buffer.

Texture unit 0 (foreground image): Mode: REPLACE
Texture unit 1 (mask): Mode: BLEND

If you want soft edges, enable framebuffer blending, and make sure you linearly filter the mask. If you want hard edges, make sure your mask also has hard edges, and use nearest filtering on the mask (or use alpha test, but that may have a larger performance impact).

See 3.7.12 in the ES 1 spec for full details.


Masking without a stencil buffer? - ThemsAllTook - Nov 14, 2008 06:50 PM

As an alternate approach, could you simply translate on the Y axis of the texture matrix? Not sure if that works in OpenGL ES, but if it does, it seems like it could accomplish this...


Masking without a stencil buffer? - Bachus - Nov 14, 2008 11:16 PM

Took some fiddling around, but the texture combiners eventually worked. Thanks guys. Smile


Masking without a stencil buffer? - arca1n - Jul 6, 2009 10:13 AM

Bachus Wrote:Took some fiddling around, but the texture combiners eventually worked. Thanks guys. Smile

How did you arrive to that solution. Can you please gimme a code snippet to point me in the right direction.


Masking without a stencil buffer? - Bachus - Jul 6, 2009 12:37 PM

arca1n Wrote:How did you arrive to that solution. Can you please gimme a code snippet to point me in the right direction.

It's basically what Frogblast said. The mask texture is white (although any color would work) and alpha-mapped. GL_REPLACE replaces the color values in the mask texture with those in the gradient texture, while keeping the alpha values. So the gradient is drawn with the alpha of the mask. Pseudo-code:

Code:
// Set up the texture and combiner
glClientActiveTexture(GL_TEXTURE0);
glActiveTexture(GL_TEXTURE0);

glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);

// Bind the mask texture and texCoords
glBindTexture(GL_TEXTURE_2D, maskTexID);
glTexCoordPointer(2, GL_FLOAT, 0, maskTexCoords);

// Set gradient texture and bind
glClientActiveTexture(GL_TEXTURE1);
glActiveTexture(GL_TEXTURE1);

glBindTexture(GL_TEXTURE_2D, gradientTexID);
glTexCoordPointer(2, GL_FLOAT, 0, gradientTexCoords);

// Draw
glVertexPointer(2, GL_FLOAT, 0, verts);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);

// Set back our old settings
glClientActiveTexture(GL_TEXTURE0);
glActiveTexture(GL_TEXTURE0);
glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);

If you're really bored (and have a few bucks), you can see the end result of this thread on the Illuminations title screen. Sneaky


RE: Masking without a stencil buffer? - mpcarr - Sep 2, 2010 01:57 PM

What do you mean by alpha-mapped?

I've got my mask texture, a white circle. and my texture i want to clip, a rainbow gradient square. I'm trying to use your technique but it just renders the mask on the screen.

any help appreciated.


RE: Masking without a stencil buffer? - Skorche - Sep 2, 2010 02:42 PM

You are probably loading your texture as RGB, you need to load an alpha texture. You can do this by passing GL_ALPHA as the internal format to glTexImage2D().