Little Known Features of VoiceOver: remapping your gestures

As a follow-up to my review of screen time, I will be looking at another little known feature of voiceover: remapping your gestures.

One of the most interesting, and least discussed, features of iOS 13 was the ability to set custom gestures for voiceover. This allowed you to remap existing gestures for things like read all, copy last spoken text, and toggle screen curtain, but it also allowed users to set gestures for entirely new voiceover capabilities, including moving by paragraph, moving between containers, and even running Siri shortcuts.

In this article, I am going to give a thorough overview of how this feature works, and make some suggestions for gestures that I have found work well in my own use cases. Please feel free to suggest alternate gesture maps in the comments below.

Using This Feature

To view the existing gesture maps and set up new gestures, go to settings, accessibility, VoiceOver, commands. From there, you will see several options for how to sort the possible gestures: all commands, touch gestures, keyboard shortcuts, handwriting, braille screen input, and the option to reset all VoiceOver commands. Clicking any of these options will sort the list based on the selected category.

From here, you have, essentially, 2 options. Clicking "all commands" will give you options to sort the entire list of gestures by further categories. The categories are interaction, basic navigation, text navigation, advanced navigation, editing, rotor, scrolling, speech and audio, output, VoiceOver, system, other, and Siri shortcuts. Clicking on any of these categories will give you a list of all possible actions within it. Clicking on one of the actions will allow you to see all keyboard, braille, braille screen input, or touch gestures associated with the given action, and the option to add a new gesture in any of the given categories.

The other option for adding a new gesture is to filter the list first by method (touch gesture, keyboard shortcut, handwriting, or braille screen input) and then by input complexity. For example, clicking on the touch gestures will give you a list, with headings, of possible actions which you may assign to a gesture.

so what does this mean?

For this example, let's assume you wanted to create a touch gesture, let's say a 3 finger swipe left and right, to move to the previous or next heading, regardless of what the rotor was set to. There are 2 ways to set this gesture up.

Method 1

Go to settings, accessibility, VoiceOver, commands, and select touch gestures. Next, find the heading that says swipe: 3 fingers. Then, flick right until you hear 3-finger swipe left and double tap it. Depending on your settings, you may also hear scroll right, because that is the associated action for the 3-finger swipe by default. It will not effect this process. After double tapping, you will be in the list to assign an action to that gesture. From here, you can either search for the word heading, or swipe down through the headings until you find ADVANCED NAVIGATION Heading and then swipe right until you find Move to Previous Heading. Double tap it to assign that gesture to the action. If the gesture already has an action associated with it, scroll right, in this case, VoiceOver will notify you of the action you are replacing and ask you to confirm that you would like to replace the action. Simply confirm to have your new gesture take effect.

Please note that until you follow the steps above by assigning the 3-finger swipe right to the next heading gesture, it will not be done by default. I would love to see more intelligent mirroring, where, after assigning a gesture to an action, iOS automatically intuits that you might wish to create a reverse action with the reverse gesture and asks if it should do so. But unfortunately, it all must be done manually at the current time.

Method 2

The second method to assign our previous heading gesture to the 3-finger swipe left is to go to settings, accessibility, VoiceOver, commands as before. However, this time, begin by clicking on all commands, followed by advanced navigation. Swipe right through the list until you find the Previous Heading Button and double tap it. Find the touch gestures heading and then flick right until you find the Add Gesture Button. After double tapping it, you will be in the list of possible touch screen gestures. Find the one that you wish to assign by either searching the list or scrolling by heading until you find the SWIPE: THREE FINGERS Heading and flick past it until you hear Three finger swipe left. After double tapping it, you will be asked if you wish to replace any existing actions assigned to this gesture. After confirming, the new gesture will be set.

The first method works well when you wish to bulk assign commands for a particular device (touchscreen, keyboard, braille display, or braille screen input) whereas the second works better if you wish to bulk add methods for a specific action.

The process for assigning keyboard and braille display gestures is much as I have described already, save that you can simply press the key or series of keys to assign a gesture rather than searching or scrolling the whole list. I would love to see something like this for the touch screen as well, but as yet it is not implemented.

the list of all possible actions

To make it easier for you to find the action you might need, I will list each action by category as Apple has listed them as well as the possible gestures you may bind.

interaction

The interaction commands deal with methods of VoiceOver interaction including activate, escape, magic tap, perform long press and secondary activate.

Basic Navigation

Navigation contains items relating to moving within and between containers, moving to the top or bottom of a document, moving to the first or last item on the screen, or moving above or below the current item. It also contains commands to move to the status bar, or between apps.

Text Navigation

Text navigation adds many features to VoiceOver including the ability to move between misspelled words, style changes, block quotes, color or font changes, and text with various attributes (bold, italic, underlined, plain text.) It also provides the ability to scroll your text by character, word, sentence and paragraph.

Rotor, Scrolling, audio, and other

Because each category here contains only one or two options, I have combined them for ease of reading. Rotor, as you might expect, provides gestures for moving between and within the rotor. Scrolling allows you to remap gestures for scrolling the screen up, down, left, or right. Speech and audio allows you to set the gesture for toggling VoiceOver speech and audio, in short, stops voiceOver from speaking while reading (1-finger tap by default), and toggling voiceOver speech (3-finger double tap by default). Other commands are those which allow copying the speech to the clipboard and toggling text selection.

output

Output provides information about interacting with the system focused item. Options include describing the item, previewing it with 3d touch, reading the item summary, reading all or from top,, reading the current character, word, line, or line count, row and column header and contents, reading the hint, reading any selected text, text style, or URL.

VoiceOver

VoiceOver gestures are those relating to Voiceover as a whole. They involve labeling an item, opening the VoiceOver settings, showing the item chooser, starting VoiceOver help, toggling the caption panel, toggling the locked modifier keys (VoiceOver modifier), toggling the screen curtain, toggling the single letter quicknav, and watching an item with VoiceOver.

System

System contains gestures for invoking the accessibility shortcut, toggling voiceOver), pressing the power button, taking a screenshot, moving between open apps, activating speak screen, pressing the home button and app switcher, as well as gestures for control center, notification center, and dock.

Other Commands

and finally, Siri shortcuts allow you to set gestures for performing any of your Siri shortcuts.

Touch Gestures

Now that we've seen all the possible things you can do, let's take a look at the possible gestures you have available to bind each action to.
Apple has made most touch gestures available with a few exceptions. The 1-finger tap and double tap must remain set to read item and activate respectively, while the 1-finger swipe left and right must remain move to previous/ next item. A 2-finger double tap and hold must remain label item. The 2-finger scrub gesture also cannot be changed, and will always act as a back gesture. You may assign a single, double, triple or quadruple tap with either 1, 2, 3, or 4 fingers. Because the 1-finger swipe let/right cannot be changed, you also have the 2-finger, 3-finger, or 4-finger swipes up, down, left or right. There are also gestures for 3-finger double tap, hold, and flick up, down, left or right, but I have never gotten these gestures to work in my testing on an iPhone.

What if I get stuck?

There are quite a lot of options available here and it can be quite easy to create a gesture scheme and realize you don't like it in practice as much as you hoped you would in theory. Fortunately, Apple has made it very easy to revert to the standard VoiceOver layout. Simply go to settings, accessibility, VoiceOver, commands, reset VoiceOver commands to restore VoiceOver gestures to factory defaults. In fact, I actually advise doing this every once in a while between tinkering with new VoiceOver gesture schemes to get a better sense of which gestures you miss when you don't have them, and evaluate your scheme against the standard one for comparison every once and a while.

Unfortunately, there are significantly more possible actions than available gestures, so every scheme is going to have some tradeoffs. But the flexibility of this new feature does allow people to prioritize the functionality that's most important to them.

So with that said, I'll outline some of the gestures I find to offer a slightly more ergonomic experience.

My Gesture Map

Braille Screen Input

I'll start with braille screen input because it is in many ways the simplest.

• 1-finger flick up (originally previous suggestion): quick action. Most often, this is the send button in various messaging apps. If you text with braille screen input, this makes sending messages almost too easy and has been responsible for more accidental texts than I'd like to admit. But it also makes texting much more comfortable for long periods of time.
• 1-finger flick down (originally next suggestion): Perform return. This one just makes sense. Its the default behavior for MBraille, and if you write emails, documents, or any kind of formatted text with braille screen input, you'll have to enter a lot of line breaks. Make that as easy as possible for yourself. This one is such a no brainer. It will make your life easier.
• 2-finger flick up/down (originally previous braille table and translate immediately.): Previous and next suggestion.

A note about these gestures: Adopting this system will also change the ways in which apps are opened on the home screen. Moving through possible apps is now done with a 2-finger swipe up or down and opening an app is done with a 1-finger swipe down.

I have opted not to change anything else, even knowing there are some functions I don't have, and some that are doubled. I would rather be able to have 2 options for things like enter and quick action, but you could opt to prioritize other gestures for additional functionality.

Touch Gestures

I have made a lot of modifications here. Feel free to take what works for you and leave the rest.

• 1-finger single tap (originally read item summary): read screen. I find simply placing 3 fingers on the screen in a quick tap to read from my place on the screen much faster than swiping down with 2 fingers.
• 2-finger swipe left (originally unassigned): Previous/next rotor. This is one of the touch gestures that I have doubled. Editing often means a lot of moving rapidly between characters, words, and lines very quickly. Even beyond that, the rotor is one of the most central experiences of using an iPhone. It should be one of the easiest and most ergonomic gestures used. I find I can rapidly switch rotor settings much easier with this gesture and that it seems to flow better with the single finger swipe up/down to move between rotor items. But I have elected to keep both this gesture and the original rotor gesture so that no matter where my hands are on the screen, at least one of them is quick.
• 3-finger swipe left/right (originally scroll left/right): Move to previous/next misspelled word. Apple has recently changed how the spellchecker is handled in iOS, and sadly, it is now much less useful. Rather than moving you between misspelled words
  , allowing you to choose a replacement, and then moving to the next misspelling, misspelled words, as it appears in the rotor, now only works to move between suggestions for already-selected misspelled words. Assigning this gesture allows VoiceOver to move to and select misspelled words, which you can replace with the misspelling option on the rotor. This does not work perfectly, but it does work. I have also had this gesture set to move between sentences and paragraphs at various points, but VoiceOver often artificially detects boundaries at the ends of sentences or lines, even when there is more in the document, limiting the usefulness of this feature at the current time.
  Note: You will have to manually switch pages on your home screen or within folders by jumping to the page scroll and swiping with one finger to change it if you do not bind something else to the scroll left/right gesture, however.
  *2-finger swipe up/down (originally read/read from top.): previous / next container. No. Really. Wanna jump to the first unread message in your list 100 percent of the time? Wanna be able to put focus directly on the first item in the settings screen and bypass extra headings and text fields? This is your gesture. Honestly, I find that I don't use next container all that often, so you could theoretically only use the 2-finger swipe up and save down for something else.
• 4-finger swipe up, down, left, and right (left/right originally next/previous app): Copy last spoken text, cut, App switcher, and paste. Nope, that's not a typo. This is where things get a little weird. Stay with me. How many times have you needed to copy text from one app to another? If its at all often, you'll enjoy this setup. There are actually 2 possible gestures for swipe up, copy, or copy last spoken text. In the latter, you will always get the text you need, though you may also need to edit out extraneous text, such as heading level, selection, etc. But, you will be able to bypass the rotor entirely. If you elect to simply use copy, you will have to use the text selection feature to highlight the exact text you want.
  The app switcher is the most efficient way to move between apps with a single gesture. Unfortunately, this setup does not allow the ability to move immediately between open apps. But it does make the app switcher much easier to get to and mitigates some of the frustration of moving between apps.

I keep trying to find a good alternate home button gesture, but currently have not been able to find one I'm satisfied with.

Note that pairing any action to the 4-finger single tap blocks the use of the 4-finger tap near the top or bottom of the screen that takes you to the first or last item on the screen.

Conclusion

Apple has only recently begun to allow custom layouts for VoiceOver. Already, though, it has greatly expanded existing functionality and enabled entirely new possibilities within VoiceOver. Locating text with specific attributes, moving between misspelled words, quickly invoking the VoiceOver settings and running Siri shortcuts are just a few things that are either now much easier or entirely new. Unfortunately, there are a few gestures which cannot be changed, and porting gesture schemes between iOS devices, even those on the same iCloud account, is not currently possible. However, even in these early days, there is much that can be done to create a custom VoiceOver experience on your iOS device, and its utility will only hopefully increase as more people become aware of this feature and Apple expands it over the coming years.

Please feel free to share your favorite new VoiceOver gestures with all our readers in the comments.

Category: Articles
jcast Friday, 03-Jan-2020 4:17 PM ET:
JazzFan_74 Saturday, 11-Jan-2020 01:12 AM ET:

You must be logged in to post comments.

Username or Email:
Password:
Keep me logged in on this computer

Or Forgot username or password?
Register for free

Tangela Mahaffey is a barely reformed English Major currently residing in Colorado. She reads entirely too many fantasy books and takes almost nothing seriously, including herself. She loves technology, music, writing, puns, and cats. She can be reached on twitter @tmmahaff