Smoothing edges with alpha mask

Member
Posts: 22
Joined: 2005.02
Post: #1
Hello,

there's a graphics glitch when blitting alpha masked textures to my quads in a 2D context. Check out the png image

in Photoshop: [Image: shot2.png]
in my game: [Image: shot1.png]

As you can see the edges are not as smooth as in Photoshop. What could be the cause for this?

My programming environment:

I created a 2D context with glOrtho
I use textured quads with blending glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)

I played around with some glTexParameter calls but this makes no difference. Any ideas? If it is necessary to post more source code I'd do that...

Thanks

Sebastian
Quote this message in a reply
Member
Posts: 567
Joined: 2004.07
Post: #2
I wouldn't sweat it; I can barely notice a difference.

It's not magic, it's Ruby.
Quote this message in a reply
Sage
Posts: 1,199
Joined: 2004.10
Post: #3
I can't say for certain without seeing your code, but it's very possible that your PNGs have premultiplied alpha -- if you're using Cocoa to load them -- and that might cause the subtle errors you're seeing.

You should dig around for the quicktime image loading code that's floating around here ( I can post my variation on it if you want ). That'll get you raw, non-premultiplied alpha.

The other issue is that Photoshop's PNG export is not too good. It's good for web, but not for OpenGL use. I can't recall what's so bad about it... but google for the SuperPNG photoshop export plugin. It's excellent.
Quote this message in a reply
Luminary
Posts: 5,143
Joined: 2002.04
Post: #4
The problem here is that photoshop uses a white background for the image. That means that the white background is visible on semitransparent parts.

There are two ways to get around this, one is to extend the colors of the opaque parts out over the transparent bits, goodness only knows how you can manage that. The other is to premultiply your image and use a premultiplied blending function. If you search the forums you'll find a thread started by Ivan which covers this in more detail.
Quote this message in a reply
Sage
Posts: 1,199
Joined: 2004.10
Post: #5
Actually, a perfectly good way to extend the colors of your image is to make a layer behind the image in photoshop that's a merged-copy of the image and run a blur on it, or a maximum/minimum filter to "spread" it out. I do this for the foliage in my game.

But then, if you use the SuperPNG exporter and use a formal alpha channel ( in the channels palette ) this problem won't happen in the first place.
Quote this message in a reply
Max
Member
Posts: 84
Joined: 2003.03
Post: #6
Hi, I'm the graphic artist of McSebi. Following Tomorrow's advice, I tried SuperPNG to create the PNGs and the results are the same. Jagged edges. I also used Apple's Preview and GraphicConverter. Got the same results. I never thought OpenGL would be so troublesome. And I thought the PlayStation 2 had the the worst graphic technology...

Freelance video game artist and video game compliance tester at Enzyme Testing Labs.
Quote this message in a reply
Member
Posts: 509
Joined: 2002.05
Post: #7
What I do is select the image, stroke 1 pixel on the outside with a color matching the outside of the image, and them stroke a 98% clear over top of those pictures. It fixed the premultiplied problem and the halo effect several of us have ran into.

But I seriously couldn't notice the difference between the two images before I read what they were.
Quote this message in a reply
Moderator
Posts: 1,140
Joined: 2005.07
Post: #8
This may be a stupid question, but I have to ask: do you have the texture filtering set to GL_LINEAR?

I can barely notice the difference, as well.
Quote this message in a reply
Max
Member
Posts: 84
Joined: 2003.03
Post: #9
akb825 Wrote:I can barely notice the difference, as well.
The aliased edges are quite noticeable when the game runs in fullscreen mode.

Freelance video game artist and video game compliance tester at Enzyme Testing Labs.
Quote this message in a reply
Moderator
Posts: 1,140
Joined: 2005.07
Post: #10
Another thing to check: do you have the GL_TEXTURE_ENV_MODE for that texture set to GL_MODULATE? If it's anti-aliased with alpha, it will carry that alpha to a white quad. I'm not sure if GL_REPLACE does alpha the same way, but I'm pretty sure that GL_DECAL is RGB only.
Quote this message in a reply
Member
Posts: 22
Joined: 2005.02
Post: #11
Ok, thanks for all your input. Here is my code which generates one texture. If I try Fix 1 everything looks great when rendered onto a black background, but all my masks have gone :-) Why that?
Fix 2 is a a bit strange because I draw the image twice but this leads to a better result. It seems drawing the image with Quartz produces the problem.

TRect is a wrapper for CGRect

Code:
    // CGImageRef bitmap; // Created from png using CGImageCreateWithPNGDataProvider

    unsigned char* bitmap_data = new unsigned char[bitmap_datasize];
    
    CGImageAlphaInfo alpha_info = (bitmap.GetDepth() == 32 ? kCGImageAlphaPremultipliedLast : kCGImageAlphaNoneSkipLast);
    CGColorSpaceRef    cg_colorspace = CGColorSpaceCreateDeviceRGB();
    CGContextRef cg_context = CGBitmapContextCreate(bitmap_data, width_, height_, 8, buffer_rowbytes, cg_colorspace, alpha_info);
    
    CGContextSetFillColorSpace(cg_context, cg_colorspace);
    CGContextSetStrokeColorSpace(cg_context, cg_colorspace);
    
// Fix 1 deactivated
//    float comp[] = { 0.0f, 0.0f, 0.0f, 1.0f };
//    CGColorRef color = CGColorCreate(cg_colorspace, comp);
//    CGContextSetFillColorWithColor(cg_context, color);
//    CGContextFillRect(cg_context, TRect(width_, height_));
//    CGContextFlush(cg_context);
    
    TRect rect(pic_width_, pic_height_);
    rect.Move(0, height_ - pic_height_);

    CGContextDrawImage(cg_context, rect, bitmap);

    // Fix 2
    for (int i = 0; i < bitmap_datasize; i += 4)
    {
        bitmap_data[i]        =  0x00;
        bitmap_data[i + 1]    =  0x00;
        bitmap_data[i + 2]    =  0x00;
    }
        
    CGContextDrawImage(cg_context, rect, bitmap);
    // Fix 2 end

    glGenTextures(1, &id_);

    glBindTexture(GL_TEXTURE_2D, id_);

    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
    glPixelStorei(GL_PACK_ALIGNMENT, 1);

    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);

    glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
    glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);

    glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);

    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width_, height_, 0, GL_RGBA, GL_UNSIGNED_BYTE, bitmap_data);
Quote this message in a reply
Moderator
Posts: 1,140
Joined: 2005.07
Post: #12
Like I said with my first suggestion, you need linear filtering. Change the GL_NEAREST to GL_LINEAR and it won't be blocky. You will only need to draw it once with that, too.
Quote this message in a reply
Sage
Posts: 1,403
Joined: 2005.07
Post: #13
You making a pac man game?

Sir, e^iπ + 1 = 0, hence God exists; reply!
Quote this message in a reply
Max
Member
Posts: 84
Joined: 2003.03
Post: #14
unknown Wrote:You making a pac man game?
Uh-ho! Busted! LOL
It's Pac the Man, you know, the most popular Pac-Man clone for the Mac. Grin

Freelance video game artist and video game compliance tester at Enzyme Testing Labs.
Quote this message in a reply
Max
Member
Posts: 84
Joined: 2003.03
Post: #15
akb825 Wrote:Like I said with my first suggestion, you need linear filtering. Change the GL_NEAREST to GL_LINEAR and it won't be blocky. You will only need to draw it once with that, too.
akb825, thanks for the solution. GL_LINEAR works perfectly. However, it blurs the graphics when our video game runs in fullscreen mode. Is there a way to disable the blur?

Freelance video game artist and video game compliance tester at Enzyme Testing Labs.
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Preventing texture from creating transparent gaps on edges ardowz 3 1,452 Jan 21, 2014 11:28 AM
Last Post: SethWillits
  Create Aquaria-like terrain- texturing edges AndyKorth 3 6,324 Jul 31, 2011 08:13 PM
Last Post: FlamingHairball
  Edge detection with sobel operator/mask g00se 5 10,849 May 20, 2011 12:23 AM
Last Post: g00se
  [SOLVED]OpenGL edges of textures mk12 2 4,429 Sep 2, 2010 08:07 PM
Last Post: mk12
  Replacing edges with degenerate quads (for shadow volumes) Coyote 9 7,806 Jan 15, 2010 07:08 PM
Last Post: Coyote