Look – no hands! How to do email handsfree – both listening and speaking

Earlier this month as we were grilled by the NZ Hi Tech Awards judges one of the judges queries whether the app is truly “hands-free”. It’s hands-free for listening to email but until recently you needed your hands if you wanted to interact with your mail.

We’ve released several new features recently that now make it possible to do email completely hands-free – including replying, forwarding, flagging, archiving or trashing.

Waiting for your command…

Now that we’ve got voice commands working nicely, we figured it would be easy to have Speaking Email listen for commands at the end of each email. Indeed, it was easy and it’s a great new feature, making it fully hands-free to interact with your email.

You can also have it wait for a command at the start of the email, after the headers – say “play” or “continue” or wait 10 seconds for it to carry on as usual and play the email body.

Bluetooth / Steering Wheel buttons

You can now use steering wheel / headset controls – next, previous and pause hardware buttons over Bluetooth or on your headphones (while technically this isn’t “hands free”, you don’t touch the phone!). We support skip next & previous email, rewind and fast forward.

If you hit the “pause” button on your steering wheel / headset, we’ve made it invoke command listening mode, so you can say any command – like “archive”, “trash” or “reply” to dictate a reply. This is released on iOS and coming soon to Android.

Bluetooth Hands-Free Profile (HFP)

We now fully support Bluetooth HFP, which means using the microphone and speakers over Bluetooth. We support Bluetooth A2DP mode also, using the device microphone and a Bluetooth speaker. When a new audio routing is detected, it displays on screen.

Speaking Email even shows the subject and sender on your Bluetooth track display

Playing email content without the clutter

Of course listening to emails works pretty nicely hands-free too, with smart content detection skipping of disclaimers, signatures, email headers, email trails that you’re in, limiting email length, and automatically moving to the next email. You may notice we even shorten common text like dates and times, http://www is shortened to dubdubdub, and we read out smiley faces (can be essential to get the mood of the writer!)

By the way you can dictate smiley faces too. Try it! 🙂

 

Get your hands on Speaking Email at https://speaking.email

Basic edition is free and premium subscribers pay a piddly $2 a month!

Advertisement

Talk back to your email

What? No voice commands?

Many users have asked for voice commands and reply dictation – in fact, they have been our most requested features. We now have both these features fully baked and released. (You may have seen these features already – we put them out and refined them over the past couple of weeks.)

It took a while to get this right: we tried two other voice engines before picking Apple and Google. On iOS we are using the newly opened up iOS voice recogniser that is part of Siri. On Android we use the built-in Google voice recognition.

We chose these because of their wide range of regional accents and languages, low battery usage, and offline usage. Most importantly we tried them and they worked pretty nicely.

Personally, I was skeptical about how useful voice would be. These features took a while to come to fruition because we wanted to do it well or not at all. Given the experiences I’ve had using software like Siri, Talkler, and trying out other voice assistants, I thought voice would be annoying to use.

Gee I’m popular, I’ve got more than 25 emails. At this point, Siri gives up…

If Siri mishears a command it just says “I didn’t understand that” and stops. No chance to try again.

How we did it

We’ve tried to address these sort of issues. I’ll give you a rundown of how it works.

Tap anywhere to see the command buttons. Then either tap a button or you can say its name, like “trash”.

You can say any of the labels on the buttons: “archive”, “trash”, “ignore”, “flag”, “repeat”, “reply”, “forward”, “play”. This makes it very easy to remember what you can say and when.

For the “forward”  command, a list of your preset forwarding recipients is shown. You can say the first word of the email address to select the person to send to. For example for “mike.nelson@beweb.co.nz” say “mike”.

Anywhere you can say a command you can say “help” to find out what you can say, or “stop” to stop listening.

Dictation too

Speaking Email dictation keeps listening as long as the operating system will allow – on iOS this is two minutes and it’s ok if you  pause for breath during this time. At this time on Android unfortunately a pause of a couple of seconds will stop the dictation (so you need to speak without pausing).

When you tap “done” or pause for 10 seconds (2 seconds on Android) Speaking Email repeats back what it thinks you said so you can change it before sending. You can say “redo”, “send”, “cancel” or “more” to add more. If you say “send” or “cancel” it carries on reading your next email to you without further ado.

The little things

We’ve taken care over the little things that make all the difference:

• Your words always appear on screen as you talk
• If you say a wrong command, we don’t leave you hanging, we just listen again
• When you say a command, we repeat it back so you know what action you triggered
• After a command we keep going, moving on and reading the next email as usual
• If you say a command like “forward” that needs more input (the recipient name) we’ve made sure the entire experience can be done by voice
• When we say commands can be done by voice, we mean without looking (not like Siri where half the time you get visual feedback only and hear nothing)

In dictation, Speaking Email uses voice recognition in the language set in your operating system settings. In case this is not what you expect, it’s shown on screen. If you have any problems with the recognition, check your system settings to make sure this language has voice recognition downloaded and available for offline use.

One satisfied user

And you know what? After my initial skepticism I now find the voice commands and dictation incredibly useful and don’t know how I got by without them!

Actually, it’s not just me, other people agree. Here’s Scott Middleton, a sales VP at Access Partners in California:

the game changer for me with your app was your new feature that allows dictating replies and functions. That is a huge advantage for Speaking Email over all the other apps…

I absolutely love the app and being an outside sales rep in the car 4 to 6 hours a day find it incredibly helpful. It is literally freeing up and hour or two of time for me every day I am on the road by having a lot less time at the desk doing email. THANKS!!!

Bonus: disclaimers

We found sometimes the dictation engine would come out with something a little off beat that we hadn’t intended. So we decided to embrace that and added a light hearted disclaimer in our tagline.

You can turn our tagline off in Preferences if you want to, and/or add your own signature.