Sign in Subscribe
Accessibility

VoiceOver vs. symbols and units

Kevin Hirsch

• 7 min read
VoiceOver vs. symbols and units

Hi, I’m Kevin, iOS developer at Immoweb, and I hate long intros. So here’s the thing…

The problem

If you check that your app works well with VoiceOver (and if you're here, I bet it's the case) and you are using symbols/units in labels, you probably have noticed that:

  1. It will be smart saying texts containing some basic units, like "12 m".
  2. It will miserably fail to say more "complex" units, like "12 m²".

Let's look at different examples and how VoiceOver handles them:

Types Example How V.O. says it How V.O. should say it
works well Ordinal numbers 2nd second [Already good]
works well Simple units 4 m 4 meters [Already good]
fails Complex units 8 m² 8 M to the power of 2 square meter
fails Formatted dates 12/05/2020 "12 slash 5 slash 2020" May 12 2020
fails Bullet points first • second first bullet second First, [break] second
fails Dashes 1 - 2 bedrooms 1, 2 bedrooms From 1 to 2 bedrooms
fails Symbols n°5 N degrees 5 Number 5
fails Abbreviation bd. BD Bedrooms

It's likely that you have a lot of those symbols/units in your apps, and you end up getting a poor VoiceOver experience.

Here are some example of symbols/units usage in our app at Immoweb (listen with sound on):

What we want is:

And we can even find the same issues in Apple's apps, like the App Store:

The solution

TL;DR

We have to use a custom property smartAccessibilityLabel instead of accessibilityLabel and replace symbols and units via regexes.
You can jump to the final code here.

The obvious accessibilityLabel

The 1st thing we could think of is the obvious. If my label is "2 bd." I would simply need to set my label.accessibilityLabel to "2 bedrooms", right? Wrong.
Well, that would work, but the main drawback with this solution is: if you need to do this in almost all your labels, you are going to end up with a bunch of boilerplate, repetitive and duplicated code. Not even counting if you need to handle labels with a mix of symbols/units, like "2 bd. • 75 m²".

What we actually want is to make VoiceOver learn some rules on how to say certain symbols/units.

Apple’s way

Let's investigate what Apple offers currentlty to solve this issue… Not much actually. Having searched through the documentation and talked with devs working in the Accessibility team at Apple, the closest thing to the feature we want is about pronunciation.

And here's what the API looks like:

let attributedString = NSAttributedString(
	string: "bd.",
	attributes: [.accessibilitySpeechIPANotation: "ˈbɛdˌɹum"]
)

label.accessibilityAttributedLabel = attributedString

Here we are:

  1. Using accessibilityAttributedLabel instead of accessibilityLabel
  2. Using NSAttributedString.Key.accessibilitySpeechIPANotation to tell that "bd." should be pronounced "bedroom" (using the International Phonetic Alphabet or IPA for short)

I think we can quickly see this is not the adequate solution because:

  • VoiceOver will ignore a given pronunciation if it doesn't match at all the original (In the example above, telling VoiceOver to say "bedroom" for "bd." won't work).
  • We have to write this code everywhere we need it (instead of once).
  • It doesn't work for more complex symbols like "1 - 2" that should be said "from 1 to 2" (same for dates like "12/04/2020").

Follow our own path (shaping our API)

We'll have to create our own solution.
What I want to end up with will look like:

label.smartAccessibilityLabel = "2 bd."

And it would do behind the scenes:

label.accessibilityLabel = "2 bedrooms"

The basis of our solution

What we want is to make accessibilityLabel smart. To make that happen, let's add a new property called smartAccessibilityLabel to NSObject:

// We extend `NSObject` because that's where `accessibilityLabel` is defined
extension NSObject {
	var smartAccessibilityLabel: String? {
		get { accessibilityLabel }
		set {
			accessibilityLabel = newValue?.improvedForReadability
		}
	}
}

extension String {
	var improvedForReadability: String {
		// We look for matches of regex pattern "bd\." to replace them with "bedrooms"
		self.replacing(pattern: "bd\\.", with: "bedrooms")
	}

	func replacing(pattern: String, with replacement: String, options: NSRegularExpression.Options = []) -> String {
		let regex = try! NSRegularExpression(pattern: pattern, options: options)
		return regex.stringByReplacingMatches(
			in: self,
			options: [],
			range: NSRange(location: 0, length: count),
			withTemplate: replacement
		)
	}
}

Now we can do:

label.smartAccessibilityLabel = "2 bd."
label.accessibilityLabel // "2 bedrooms"

// You can also call it this way
label.accessibilityLabel = "2 bd.".improvedForReadability

Handling multiple symbols/units

Here are the next steps:

  1. We want to support all the other symbols/units we can have in our apps within improvedForReadability variable.
  2. We want to support of several symbols/units in the same string (e.g. "2 bd. • 50 m²").

For that, we'll change improvedForReadability implementation to:

var improvedForReadability: String {
	// List all of the substitutions we want to make
	let substitutions = [
		// tuples of (pattern to match, what to replace it with)
		("bd\\.", "bedrooms"),
		("m²", "square meters"),
		("•", ", ")
		// Add a more
	]

	return substitutions.reduce(self) { (string, substitution) in
		let (pattern, replacement) = substitution
		return string.replacing(pattern: pattern, with: replacement)
	}
}

Now we can do:

label.smartAccessibilityLabel = "2 bd. • 50 m²"
label.accessibilityLabel // "2 bedrooms, 50 square meters"

And it works 🙌
Now let's continue on improving our solution!

More complex symbols

They are a few more complex cases where you need to take variables into account. For example, replacing "1 - 3" to "from 1 to 3" will require us to use regex capture groups to capture the variables (here the left hand side and right hand side of the "-") and reference them into the replacement string.

Here's what it looks like:

var improvedForReadability: String {
	let substitutions = [
		…,
		("(\\d+) - (\\d+)", "from $1 to $2")
	]

	…
}

Now we can do:

label.smartAccessibilityLabel = "1 - 3"
label.accessibilityLabel // "from 1 to 3"

To engloble more cases, we could support in addition to simple digits:

  • Formatted digits with ,, . or even spaces (e.g. "1 250,25" in European format or "1,250.25" in US format)
  • Hours (e.g. "18:45")

Then we would need to change the \d+ regex to:

var improvedForReadability: String {
    // `(?:)` is a non capturing group, so we won't be able to reference it with $n
	let number = #"\d(?:[\.,:\s]?\d)*"#

	// List all of the substitutions we want to make
	let substitutions = [
		…,
		("(\(number)) - (\(number))", "from $1 to $2")
	]

	…
}

Now it will handle strings like:

  • "08:00 - 09:00"
  • "10 000 - 20 000,5"
  • "10,000 - 20,000.5"

All the symbols

Here are some examples of other symbols we handle in the Immoweb app:

let substitutions = [
	("(CO²|CO₂)", "CO2"),
	("kWh", "Kilowatt hour"),
	("(n|N)°", "number"),
    ("(\(number)) ?/ ?(\(number))", "$1 of $2"),
    ("/", " per "),
	("(+/-|+-|±)", "more or less")
]

Improve performance

Since we are dealing with regexes, it can become quite heavy and make the app slower if done on the main thread. So to be sure not to slow down the app unnecessarily:

  • We'll use smartAccessibilityLabel only when there's a chance we need it. For basic stuff, we'll still use accessibilityLabel.
  • We'll call improvedForReadability on a background thread if none of UIAccessibility features are enabled.

Here's what it looks like:

extension NSObject {
	var smartAccessibilityLabel: String? {
		get { accessibilityLabel }
		set {
			setInBackgroundIfAccessibilityOff(
				\.accessibilityLabel,
				to: newValue?.improvedForReadability
			)
		}
	}

	func setInBackgroundIfAccessibilityOff(
		_ keyPath: ReferenceWritableKeyPath<NSObject, String?>,
		to value: @autoclosure @escaping () -> String?
	) {
		if UIAccessibility.isVoiceOverRunning
			|| UIAccessibility.isSpeakScreenEnabled
			|| UIAccessibility.isSwitchControlRunning
		{
			self[keyPath: keyPath] = value()
		} else {
			DispatchQueue.global(qos: .background).async { [weak self] in
				let generatedValue = value() // In case it takes time, it's done in background
				DispatchQueue.main.async {
					// Since it's about UI, we do this on the main thread, just to be sure ;)
					self?[keyPath: keyPath] = generatedValue
				}
			}
		}
	}
}

Final code

Link to the gist.

extension NSObject {
	// The same can be done for `accessibilityValue` and `accessibilityHint` if needed.
	var smartAccessibilityLabel: String? {
		get { accessibilityLabel }
		set {
			setInBackgroundIfAccessibilityOff(
				\.accessibilityLabel,
				to: newValue?.improvedForReadability
			)
		}
	}

	private func setInBackgroundIfAccessibilityOff(
		_ keyPath: ReferenceWritableKeyPath<NSObject, String?>,
		to value: @autoclosure @escaping () -> String?
	) {
		if UIAccessibility.isVoiceOverRunning
			|| UIAccessibility.isSpeakScreenEnabled
			|| UIAccessibility.isSwitchControlRunning
		{
			self[keyPath: keyPath] = value()
		} else {
			DispatchQueue.global(qos: .background).async { [weak self] in
				let generatedValue = value() // In case it takes time, it's done in background
				DispatchQueue.main.async {
					// Since it's about UI, we do this on the main thread, just to be sure ;)
					self?[keyPath: keyPath] = generatedValue
				}
			}
		}
	}
}

private let number = #"\d(?:[\.,:\s]?\d)*"#

extension String {
	var improvedForReadability: String {
		// List all of the substitutions we want to make
		// Array of tuples of (pattern to match, what to replace it with)
		let substitutions = [
			("/", " per "),
			("(CO²|CO₂)", "CO2"),
			("kWh", "Kilowatt hour"),
			("(n|N)°", "number"),
			("(\(number)) ?/ ?(\(number))", "$1 of $2"),
			("(+/-|+-|±)", "more or less")
		]

		return substitutions.reduce(self) { (string, substitution) in
			let (pattern, replacement) = substitution
			return string.replacing(pattern: pattern, with: replacement)
		}
	}

	func replacing(pattern: String, with replacement: String, options: NSRegularExpression.Options = []) -> String {
		let regex = try! NSRegularExpression(pattern: pattern, options: options)
		return regex.stringByReplacingMatches(
			in: self,
			options: [],
			range: NSRange(location: 0, length: count),
			withTemplate: replacement
		)
	}
}

Conclusion

We discovered that VoiceOver does not always work well out of the box with symbols and units. Unfortunately, for the moment, Apple does not offer an adequate solution. Even in their own apps like the App Store, VoiceOver struggles to say clearly symbols and units. Apps in general tend to have at least a few symbols/units/abbreviations/text decorations. We can make VoiceOver learn how to read those with some homemade code, by creating and using a smartAccessibilityLabel instead of the native accessibilityLabel where it's needed.

Thank you for reading!
See you soon for some new accessibility adventures 👋

Sign up for more like this.

Enter your email
Subscribe