Some people are complaining about some Mac OS X applications which do not follow their preference for subpixel anti-aliasing. Pierre Igot thinks this is due to incompetence. Michael Tsai speculates that there is a hidden issue that makes Quartz choose standard antialiasing. John Gruber just find this curious. And I know exactly why this happens and why it isn’t easy to fix.
Michael Tsai thinks this is linked to off-screen buffers which then got composited together:
The pattern I see is that applications like Pages and OmniGraffle draw multiple layers with possibly overlapping elements. A logical way to implement this is to draw the elements separately into off-screen buffers and then composite them together.
And he is exactly right. The problem is that these buffers have a transparent background on which Quartz never apply subpixel antialiasing. And that’s not because of incompetence, it is because it is simply impossible to draw subpixel antialiasing on a transparent background. The culprit is called aRGB, can you see it?
If not, here is the explanation.
Subpixel antialiasing works by giving different values to the three subpixels inside a pixel (red, green, blue) based on how far each of them are from the mathematical outline of the character’s glyph. When drawing, the compositor must apply each subpixel’s values with a different level of transparency.
The “a” in aRGB stands for alpha — it’s the transparency value for the whole pixel. That’s it, the whole pixel: when aRGB pixels are later composited with other layers, each subpixel is mixed with the same level of transparency. This is incompatible with subpixel antialiasing which requires a different transparency level for each of the three subpixels.
That said, I think Quartz is pretty smart because subpixel antialiasing works on semitransparent backgrounds, like menus pulling out from the menu bar. It implements some sort of gradual degradation of the subpixels. For instance, if the background on which it draw text has a transparency of 50%, half of each subpixel value will be standard antialiasing, the other half being subpixel antialiasing.
So if an application draws in an offscreen buffer with a transparent background — surprise! — no subpixel antialiasing. Until Apple implements some kind of aRaGaB compositor, application developers will have to choose between a fast application with off-screen rendering or subpixel antialiasing, but not both. I think the choice is easy to make for developers.
A hypothetical aRaGaB compositor wouldn’t have acceleration from the graphic card anyway (graphic cards use aRGB); proper subpixel transparency may not be enough of an improvement to justify a slower and more RAM-hungry compositor.
Subpixel antialiasing does not work for shadowy text either probably because Quartz internals render the text in an off-screen buffer to compute the shadow.
You can test and see partial subpixel antialiasing for semitransparent backgrounds by playing with the transparency of a terminal window over another window of the same color. When the background becomes completely transparent, you’ll see standard anti-aliasing.