Understanding multisampled FBOs

Member
Posts: 35
Joined: 2009.10
Post: #1
Just thinking about how to implement multisampled FBOs.

As far as I can tell, creating and using the FBO itself is largely the same. However, I have to use render buffers instead of textures for colour, where the render buffers are created using glRenderbufferStorageMultisampleEXT.

In order to get the contents of the multisampled FBO to a texture I bind a second FBO using regular textures instead of multisampled render buffers using GL_READ_FRAMEBUFFER_EXT instead of the usual GL_FRAMEBUFFER_EXT. Then I bind the multisampled FBO using GL_DRAW_FRAMEBUFFER_EXT. Finally I call glBlitFramebufferEXT to transfer the colour information from the multisampled FBO to the regular (renderable) FBO.

Now on to the things I'm unclear about. First, does the render buffer I use for depth also have to be multisampled in the multisampled FBO? Second, does this handle multiple colour attachments well? As in, is the only thing I have to do is give the render FBO the same number of colour attachments (as textures) as the multisampled FBO (as render buffers)?
Quote this message in a reply
Sage
Posts: 1,234
Joined: 2002.10
Post: #2
Coyote Wrote:First, does the render buffer I use for depth also have to be multisampled in the multisampled FBO?

Yes. Read the documentation about FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_EXT.

Quote: Second, does this handle multiple colour attachments well? As in, is the only thing I have to do is give the render FBO the same number of colour attachments (as textures) as the multisampled FBO (as render buffers)?

Multiple multisampled color attachments are possible as long as they all have the same number of samples (and, with EXT_fbo, the same internal format and dimensions.)
However, when you resolve the attachments via a blit, you can only blit a single attachment at a time. The blit is defined (see issue 12) to only source from one read buffer. The result will be replicated to all draw buffers, which isn't generally useful. So you need one blit per attachment, setting the read/draw buffer as you go.
Quote this message in a reply
Member
Posts: 35
Joined: 2009.10
Post: #3
arekkusu Wrote:So you need one blit per attachment, setting the read/draw buffer as you go.

This took a while to click for me. So, I basically need code like this then?
Code:
for (int i = 0; i < numTextures; ++i)
{
    glBindFramebufferEXT(GL_READ_FRAMEBUFFER_EXT, multisampledFBO);
    glReadBuffer(GL_COLOR_ATTACHMENT0_EXT + i);

    glBindFramebufferEXT(GL_DRAW_FRAMEBUFFER_EXT, fbo);
    glDrawBuffer(GL_COLOR_ATTACHMENT0_EXT + i);

    glBlitFramebufferEXT( 0, 0, texWidth, texHeight, 0, 0, texWidth, texHeight, GL_COLOR_BUFFER_BIT, GL_LINEAR);
}

This is hard to test right now though because, at some point, using an FBO with multisampled render buffers started causing kernel panics. Cry

I'm not sure how useful this'll be, but here's the code for my FBO class. The general idea is that it keeps one FBO with regular texture colour attachments and, if given a number of samples greater than zero, a second FBO with multisampled renderbuffer colour attachments. The method blitToTextures copies the contents of the multisampled FBO to the regular FBO if applicable.

Code:
//
//  FramebufferObject.h
//

#import <Cocoa/Cocoa.h>

@interface FramebufferObject : NSObject
{
    GLuint fbo, depthbuffer;
    GLuint multisampledFBO, multisampledDepthBuffer;

    int numSamples;

    int numTextures;
    GLuint *textures;
    GLuint *multisampledColourBuffers;

    double scale;
    GLsizei texWidth, texHeight;
}

@property(readonly) GLsizei texWidth, texHeight;

+ (void)unbind;

- (id)initWithNumberOfTextures:(int)n;
- (id)initWithNumberOfTextures:(int)n withDepthBuffer:(BOOL)wantDepth;
- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s;
- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s
   withDepthBuffer:(BOOL)wantDepth;

// Multisampling
// For now, assume that if I want a multisampled FBO, I also want it
// to be fullscreen and with a depth buffer.
- (id)initWithNumberOfTextures:(int)n withSamples:(int)samples;
- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s
   withDepthBuffer:(BOOL)wantDepth withSamples:(int)samples;

- (BOOL)isMultisampled;
- (BOOL)useDepthBuffer;

- (void)resizeWithWidth:(GLsizei)width withHeight:(GLsizei)height;

- (void)bind;

- (void)blitToTextures;

- (GLuint)getTextureAtIndex:(int)index;
- (NSNumber *)getTextureAsNumberAtIndex:(int)index;

@end
Code:
//
//  FramebufferObject.m
//

#import "FramebufferObject.h"

static void checkFBOStatus()
{
    GLenum status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
    switch (status)
    {
        case GL_FRAMEBUFFER_COMPLETE_EXT:
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT_EXT");
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT_EXT");
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT");
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_FORMATS_EXT");
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_DRAW_BUFFER_EXT");
            break;
        case GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_INCOMPLETE_READ_BUFFER_EXT");
            break;
        case GL_FRAMEBUFFER_UNSUPPORTED_EXT:
            NSLog(@"FBO error: GL_FRAMEBUFFER_UNSUPPORTED_EXT");
            break;
        default:
            NSLog(@"FBO error: Unknown error");
            break;
    }

}

//--------------------------------------------------------------------

@implementation FramebufferObject

@synthesize texWidth, texHeight;

+ (void)unbind
{
    glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);
}

- (id)initWithNumberOfTextures:(int)n
{
    return [self initWithNumberOfTextures:n scaleSizeBy:1.0
      withDepthBuffer:YES withSamples:0];
}

- (id)initWithNumberOfTextures:(int)n withDepthBuffer:(BOOL)wantDepth
{
    return [self initWithNumberOfTextures:n scaleSizeBy:1.0
      withDepthBuffer:wantDepth withSamples:0];
}

- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s
{
    return [self initWithNumberOfTextures:n scaleSizeBy:s
      withDepthBuffer:YES withSamples:0];
}

- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s
   withDepthBuffer:(BOOL)wantDepth
{
    return [self initWithNumberOfTextures:n scaleSizeBy:1.0
      withDepthBuffer:YES withSamples:0];
}

- (id)initWithNumberOfTextures:(int)n withSamples:(int)samples
{
    return [self initWithNumberOfTextures:n scaleSizeBy:1.0
      withDepthBuffer:YES withSamples:samples];
}

- (id)initWithNumberOfTextures:(int)n scaleSizeBy:(double)s
   withDepthBuffer:(BOOL)wantDepth withSamples:(int)samples
{
    self = [super init];
    if (self)
    {
        scale = s;

        glGenFramebuffersEXT(1, &fbo);

        numSamples = samples;

        if ([self isMultisampled])
        {
            glGenFramebuffersEXT(1, &multisampledFBO);
        }
        else
        {
            multisampledFBO = 0;
        }

        if (wantDepth)
        {
            if ([self isMultisampled])
            {
                glGenRenderbuffersEXT(1, &multisampledDepthBuffer);
                depthbuffer = 0;
            }
            else
            {
                glGenRenderbuffersEXT(1, &depthbuffer);
                multisampledDepthBuffer = 0;
            }
        }


        numTextures = n;

        if ([self isMultisampled])
        {
            multisampledColourBuffers = calloc(numTextures,
              sizeof(*multisampledColourBuffers));
            glGenRenderbuffersEXT(numTextures,
              multisampledColourBuffers);
        }
        else
        {
            multisampledColourBuffers = 0;
        }


        textures = calloc(numTextures, sizeof(*textures));
        glGenTextures(numTextures, textures);

        [self resizeWithWidth:1 withHeight:1];
    }

    return self;
}

- (BOOL)isMultisampled
{
    return numSamples > 0;
}

- (BOOL)useDepthBuffer
{
    return (depthbuffer > 0) || (multisampledDepthBuffer > 0);
}

- (void)resizeWithWidth:(GLsizei)width withHeight:(GLsizei)height
{
    texWidth = (GLsizei)((double)width * scale);
    texHeight = (GLsizei)((double)height * scale);

    if (texWidth < 1)
    {
        texWidth = 1;
    }
    if (texHeight < 1)
    {
        texHeight = 1;
    }

    [self bind]; // Bind to multisampled FBO if multisampled, bind to
                 // regular FBO otherwise.

    // Depth buffer

    if ([self useDepthBuffer])
    {
        if ([self isMultisampled])
        {
            glBindRenderbufferEXT(GL_RENDERBUFFER_EXT,
              multisampledDepthBuffer);

            // We asked for a 32-bit depth buffer
            glRenderbufferStorageMultisampleEXT(GL_RENDERBUFFER_EXT,
              numSamples, GL_DEPTH_COMPONENT32, texWidth, texHeight);

            glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT,
              GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT,
              multisampledDepthBuffer);
        }
        else
        {
            glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, depthbuffer);

            // We asked for a 32-bit depth buffer
            glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT,
              GL_DEPTH_COMPONENT32, texWidth, texHeight);
            
            glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT,
              GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT,
              depthbuffer);
        }

        glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0);
    }

    // Colour buffers

    if ([self isMultisampled])
    {
        for (int i = 0; i < numTextures; ++i)
        {
            glBindRenderbufferEXT(GL_RENDERBUFFER_EXT,
              multisampledColourBuffers[i]);

            glRenderbufferStorageMultisampleEXT(GL_RENDERBUFFER_EXT,
              numSamples, GL_RGB, texWidth, texHeight);

            glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT,
              GL_COLOR_ATTACHMENT0_EXT + i, GL_RENDERBUFFER_EXT,
              multisampledColourBuffers[i]);
        }
        glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0);

        checkFBOStatus();

        // Now bind to texture FBO to create colour textures
        glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo);
    }

    for (int i = 0; i < numTextures; ++i)
    {
        glBindTexture(GL_TEXTURE_2D, textures[i]);
        // Depends on GL_ARB_texture_non_power_of_two
        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, texWidth, texHeight,
          0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, NULL);

        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER,
          GL_LINEAR);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER,
          GL_LINEAR);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);

        glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT,
          GL_COLOR_ATTACHMENT0_EXT + i, GL_TEXTURE_2D, textures[i],
          0);
    }
    glBindTexture(GL_TEXTURE_2D, 0);

    checkFBOStatus();

    [FramebufferObject unbind];
}

- (void)bind
{
    if ([self isMultisampled])
    {
        glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, multisampledFBO);
    }
    else
    {
        glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo);
    }
}

- (void)blitToTextures
{
    if ([self isMultisampled])
    {
        for (int i = 0; i < numTextures; ++i)
        {
            glBindFramebufferEXT(GL_READ_FRAMEBUFFER_EXT,
              multisampledFBO);
            glReadBuffer(GL_COLOR_ATTACHMENT0_EXT + i);

            glBindFramebufferEXT(GL_DRAW_FRAMEBUFFER_EXT, fbo);
            glDrawBuffer(GL_COLOR_ATTACHMENT0_EXT + i);

            glBlitFramebufferEXT( 0, 0, texWidth, texHeight,
              0, 0, texWidth, texHeight, GL_COLOR_BUFFER_BIT,
              GL_LINEAR);
        }

        [FramebufferObject unbind];
    }

}

- (GLuint)getTextureAtIndex:(int)index
{
    GLuint result = 0;

    if ((index >= 0) && (index < numTextures))
    {
        result = textures[index];
    }

    return result;
}

- (NSNumber *)getTextureAsNumberAtIndex:(int)index
{
    return [NSNumber
      numberWithUnsignedInt:[self getTextureAtIndex:index]];
}

- (void)dealloc
{
    glDeleteFramebuffersEXT(1, &fbo);

    if (multisampledFBO > 0)
    {
        glDeleteFramebuffersEXT(1, &multisampledFBO);
    }

    if (depthbuffer > 0)
    {
        glDeleteRenderbuffersEXT(1, &depthbuffer);
    }
    if (multisampledDepthBuffer > 0)
    {
        glDeleteRenderbuffersEXT(1, &multisampledDepthBuffer);
    }

    if (multisampledColourBuffers)
    {
        glDeleteRenderbuffersEXT(numTextures,
          multisampledColourBuffers);
        free(multisampledColourBuffers);
    }

    glDeleteTextures(numTextures, textures);
    free(textures);

    [super dealloc];
}

@end

The FramebufferObject in action:
Code:
FramebufferObject *sceneFBO;

...

// If I don't include "withSamples:4", I get no kernel panics
sceneFBO = [[FramebufferObject alloc] initWithNumberOfTextures:NUM_SCENE_TEXTURES withSamples:4];

...

- (void)resizeWithWidth:(int)newWidth withHeight:(int)newHeight
{
    [sceneFBO resizeWithWidth:newWidth withHeight:newHeight];
    ...
}

...

- (void)render
{
    [sceneFBO bind];

    // Bind shader
    ...

    glDrawBuffers(NUM_MULTITEX_BUFFERS, MULTITEX_BUFFERS);

    // Draw scene
    ...

    [sceneFBO blitToTextures];

    [FramebufferObject unbind];

    // Get texture with [sceneFBO getTextureAtIndex:texIndex]
    ...
}
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #4
Userland code should not be able to panic your machine. File a bug. http://bugreport.apple.com/
Quote this message in a reply
Member
Posts: 35
Joined: 2009.10
Post: #5
Good call, though it seems I'll have to sign up for an ACD membership. This isn't a bad thing, but it is another online account I'll have to keep track of. Plus I'll have to figure out how to effectively isolate my code into something I feel comfortable sending to Apple. Sneaky

The reason I'm dealing with multisampled FBOs in the first place is because I'm using FBOs for a bloom effect, where I render have a texture with the regular scene and a texture with only the glowing parts of the scene which I'd blur. I'm thinking of abandoning multisampled FBOs altogether and just rendering my scene twice, once to the screen and once to an FBO that will just contain the glowing parts. For the glow texture I'd set the alpha to zero to block parts that don't glow. When I'm finished rendering and blurring the glow texture I'd draw that over the screen. Does this sound like a good plan?
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #6
You can attach a binary to your bug report rather than source if you prefer. Source is better than a binary, but a binary is better than nothing...
Quote this message in a reply
Sage
Posts: 1,234
Joined: 2002.10
Post: #7
Coyote Wrote:Does this sound like a good plan?

Is there a reason you can't use the destination alpha to store the "bloom factor"? For opaque objects, you can coalesce two rendering passes and only use a single attachment.
Quote this message in a reply
Member
Posts: 35
Joined: 2009.10
Post: #8
arekkusu Wrote:Is there a reason you can't use the destination alpha to store the "bloom factor"?
Yes: I have no idea how I'd use that. Blink

All the resources I've read talk about doing bloom by rendering the whole scene on one texture and rendering the glowing parts on another texture. I'd blur the glow texture by rendering it to another texture with a horizontal blur shader, then render the second glow texture to the first glow texture with a vertical blur shader. After I have my final blurred glow texture I'd render it and the scene texture together using additive blending (though I'm actually using screen blending).

If I use a single attachment and, say, set the alpha to one for every fragment that's part of a glow colour and to zero for every fragment that's part of a regular colour, how exactly would I blur that so that the glow colours bleed into the regular colours?

Also, I just tried my earlier idea tonight. I got it almost working, but I couldn't figure out how to set the alpha of fragments with some blurred colour, so I ended up with bloom regions with black outlines. Annoyed
Quote this message in a reply
Sage
Posts: 1,234
Joined: 2002.10
Post: #9
If you want a blue sky to "bloom" red or some other color, then you'll need two color attachments. But if you just blur the regular color and add a fraction of it to the original, it'll look like bloom.

So render the scene, writing a bloom factor [0..1] for every fragment into the destination alpha.
Then take the resulting RGBA buffer, downsample it, do a separable gaussian on it, whatever you want to blur it. Scale the result by the alpha, add it back to the original.
Quote this message in a reply
Member
Posts: 35
Joined: 2009.10
Post: #10
arekkusu Wrote:If you want a blue sky to "bloom" red or some other color, then you'll need two color attachments. But if you just blur the regular color and add a fraction of it to the original, it'll look like bloom.
Basically the effect I'm after is having certain polygons glow, which sound like the second effect you described.

Quote:So render the scene, writing a bloom factor [0..1] for every fragment into the destination alpha.
Are we rendering to the screen or to a texture? Because if I'm setting the alpha component wouldn't I have to disable blending to make sure I don't make objects by mistake? Though this isn't a big deal, since I don't think I'll have transparent objects in my game anyway.

Quote:Then take the resulting RGBA buffer, downsample it, do a separable gaussian on it, whatever you want to blur it. Scale the result by the alpha, add it back to the original.
So am I basically copying the scene from the screen to a texture, processing that texture, then rendering the texture back over the screen?

In any case, I've redicovered the use of glBlendFunc. A lack of understanding it is likely what caused me trouble when I tried rendering a texture over an existing scene.
Quote this message in a reply
Sage
Posts: 1,234
Joined: 2002.10
Post: #11
Coyote Wrote:Are we rendering to the screen or to a texture?
That's up to you. Either way works, it is just a tradeoff in functionality/performance. If you render to the screen, you get implicit multisample resolve, but you can't texture directly from the result. If you render to a texture you need to explicitly handle multisample resolve, but you can then texture directly from the result.

Quote:Because if I'm setting the alpha component wouldn't I have to disable blending
Yes, which is why I said "for opaque objects" earlier. If you want translucent objects that also glow, you'll either need two rendering passes or two render targets, or use exactly the same value for "bloom factor" and "opacity".

Quote:So am I basically copying the scene from the screen to a texture, processing that texture, then rendering the texture back over the screen?
That's the basic idea for any fullscreen image processing, but render-to-texture is intended to eliminate the need for copying the screen.

There's several writeups on implementing this type of bloom, for example at gamasutra or GDC (see "postprocessing".) You've got the basic idea already. I am merely suggesting a single-channel "bloom factor" can be conveniently stored in the destination alpha of the render target you're already using.
Quote this message in a reply
Member
Posts: 35
Joined: 2009.10
Post: #12
Thank you. This all should be enough to get me going. Just one more thing though:

arekkusu Wrote:If you render to a texture you need to explicitly handle multisample resolve, but you can then texture directly from the result.
How would I manually do the multisampling for the texture, assuming I don't want to use multisampled FBOs?

Thanks again guys.
Quote this message in a reply
Sage
Posts: 1,234
Joined: 2002.10
Post: #13
Without EXT_framebuffer_multisample, you can manually supersample-- create a texture up to two times larger than you want, and render to it. Manually downsample to the real size.

That's not as efficient, since you burn fill rate on every sample. And it is lower quality, since you can't use rotated sample positions like multisampling.
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Draw to texture using FBOs - aspect ratio issues Madrayken 2 4,387 Jul 15, 2010 11:47 AM
Last Post: Madrayken
  NPOT FBOs in OpenGL 2.0 cjcaufield 5 4,182 Jun 22, 2009 11:23 PM
Last Post: cjcaufield
  Understanding glColorMaterial PhysicsGuy 3 5,944 Nov 25, 2008 06:25 PM
Last Post: PhysicsGuy
  Trouble with glCopyTexSubImage2D when using multisampled rendering TomorrowPlusX 11 5,382 Nov 1, 2006 02:54 PM
Last Post: OneSadCookie
  Minor issue with understanding ExitToShell 15 6,353 Jul 13, 2005 10:22 AM
Last Post: MattDiamond