Font substitution and missing text

I recently found a bug in my work-in-progress-next-major-version of Counterparts Lite where text in certain languages was not showing up in the text editor. The problem ended up being related to font substitution. If characters from certain languages aren’t showing up in your Cocoa text views, here’s something to check…

The first thing to understand is that no font contains all the characters defined in Unicode. When the text system encounters characters that aren’t representable in the current font, it chooses another font that has the correct glyphs to display those characters.

Every time you change some part of a NSTextStorage object in Cocoa, its processEditing method is called to optimize attribute ranges and fix things that could be incoherent (such as a paragraph style spanning only part of a paragraph). This is also where font substitution happens.

You can observe what processEditing is doing easily by setting a delegate on the text storage implementing the willProcessEditing and didProcessEditing methods. After changing the text, printing the text storage’s content in willProcessEditing will result in this single text section with uniform attributes:

Hello 漢字{
    NSColor = "Catalog color: System textColor";
    NSFont = "\"Helvetica 12.00 pt. P [] (0x7f85d9f60070)
        fobj=0x7f85d9f4c320, spc=3.33\"";

And then in didProcessEditing you will see it was split in two text sections with different font attributes:

Hello {
    NSColor = "Catalog color: System textColor";
    NSFont = "\"Helvetica 12.00 pt. P [] (0x7fbdc15a25c0)
        fobj=0x7fbdbec6a0d0, spc=3.33\"";
    NSColor = "Catalog color: System textColor";
    NSFont = "\"HiraginoSans-W3 12.00 pt. P [] (0x7fbdc15b2330)
        fobj=0x7fbdc15adbe0, spc=4.00\"";

The font was changed for the characters not representable in Helvetica to ensure they can still be shown. Font substitution at work!

Perhaps however our app wants to control the font in the text view in order to replace whatever style could come from the user pasting some text. Those delegate methods sure seem like a good place to do that, and they are.

It is important to know however that there’s a crucial difference between changing the font in willProcessEdting and in didProcessEditing:

  • willProcessEditing is called before font substitution. Any font you set while in willProcessEditing will get substituted with an appropriate font for characters not available in the font you choose.

  • didProcessEditing is called after font substitution. If you change the font while in didProcessEditing there will be no further font substitution, and thus you should make sure the font you set can display all the characters.

In summary, if you have to change the font after the text storage was edited, do it in willProcessEditing and it’ll do the right thing. Don’t do it in didProcessEditing.

It’s also important to note that you can change the font in the text storage without manipulating the text storage directly (such as in textView.font = myFont). The same rule apply: do it in willProcessEditing because if you do it from didProcessEditing no font substitution will occur and you can end up with characters not showing up.

macOS Catalina: Authorize Screen Recording

macOS Catalina brings tighter security in various ways. One change is about preventing apps from seeing what’s on your screen. Apps that want to do that now require a special authorization from the user. This includes two of my apps: Sim Daltonism and Red Stripe.

The first time you open the app, macOS will tell you it wants to record the screen and will offer to open System Preferences. Until authorized in System Preferences, all the app can see is your desktop picture.

You can open System Preferences from the alert shown by macOS when you first open the app, which will bring you exactly at the right place. Otherwise, you can go there by choosing System Preferences from the Apple menu: from there, open the Security & Privacy pane, choose the Privacy tab, and select the the Screen Recording category.

You will see a list of apps that requested the ability to record the screen. Check the ones you want to authorize. You might be prompted to quit the app so the setting takes effect; this is unnecessary in the case of Sim Daltonism and Red Stripe but might be needed for other apps.

Black Light Pro 1.1 on the App Store

The short version of this post is that Black Light Pro 1.1 is out, and is now available on the Mac App Store. If you’re interested in the story behind this, read below.

Last year when I was developing Black Light Pro, I intended it to be available on the App Store as well as directly from my website, like all my other apps. Things don’t always work as intended though. The App Store is famous for its strict sandboxing requirements, disallowing access to many functions of the OS in the name of security. One of these is for changing the current color profile of the screen.

There’s two ways really the operating system lets us affect how colors are rendered on a screen:

  1. You can directly change the display transfer table and it’ll have an instantaneous effect. It’s a bit fragile too because many things can reset this transfer table unexpectedly, like the display waking from sleep, plugging a secondary screen, other apps quitting, etc.

  2. You can change the color profile of a screen. The color profile tells many things about a screen, one of them is the display transfer table the operating system should use. When the transfer table applied in method #1 gets reset, it becomes whatever is in the screen’s color profile. This makes the profile the ideal place to apply a long-lasting effect.

In Black Light I always used method #1. For Black Light Pro I wanted to use #2 to make the effect more robust by changing the screen’s profile while the app is running. This is better because it avoids the flashes of “normal” colors that happen whenever the transfer table is reset.

So, the sandbox got in the way. Since you can’t change the color profile of a screen from a sandboxed app, that feature was out for the Mac App Store version. The app would still work inside the sandbox, but changing the profile wouldn’t. I didn’t want to remove the feature (because it truly makes things better), and I did not want to make one version sandboxed and the other not for various reasons. In the end, I released it only on my website.

A few months ago, it came to my attention that Black Light Pro would sometimes crash unexpectedly. This would manifest by the app disappearing from the menu bar and scheduled effects no longer triggering. From the crash logs it looked related to the code for changing the profile.

I couldn’t figure out exactly how to reproduce it, but it seems like a reference counting problem with a value returned from one of the function calls needed to change the profile. Not sure if I could do something about it (the bug could be on Apple’s side), I decided to move the profile-management code to a separate process: if that separate process crashes, then a new one could be re-spawned and continue where the other left. It’s fault-tolerant now.

This complicated bug fix has an interesting effect: it makes it possible to sandbox the main process. If the main process is sandboxed, that means it can go on the Mac App Store… as long as the profile management process is not included. And that’s what I’ve done for 1.1.

I’m a bit proud now to announce Black Light Pro is now available on the App Store. Also the color profile functionality is more robust now in the direct version.

Gamma Board 1.3: Presets

There’s a new version of Gamma Board out today with new features helping you color-manage screens from a distance. Three highlights:

  • Presets are a new feature allowing you to save and restore settings for screens and groups. Tap the sliders icon in the setting panel to create or apply a new preset.

  • Propagation of a group settings to all the screen settings. Tap the group icon at the top of the group settings and you’ll be offered to merge the group settings into each individual screen settings. You can also reset individual settings for all the screens in the group in the same way.

  • The document browser has been changed to use the standard iOS document browser (if running on iOS 11 or later). You can now save set documents on iCloud, locally on the device, or using other storage services.

Gamma Board 1.3 also includes a couple of minor bug fixes.

You can download it from the App Store for $79.99 USD — a 20% reduction. Or request a custom branded version for your organization.

  • © 2003–2023 Michel Fortin.