Update 23rd June:

Since writing this blog I see that the Microsoft Garage have also released a new product called MS Dictate which is a plugin for Outlook, Word and Powerpoint that allows you to dictate text using the same speech-to-text engine used by Cortana. You can download it for free here.

Recently I’ve been working with a partner that has a school for deaf and hearing impaired students as a customer. It’s been really interesting exploring how technology is used in environments like this, where the need for video communication to enable sign language is paramount.

As a result, I’ve been digging into the accessibility options within Microsoft’s Windows 10 and Office365 products and it has reinforced the observations I’ve noticed already since joining Microsoft at the start of the year: accessibility and inclusive design really is at the heart of all Microsoft products.

Accessibility is a priority for Microsoft for three key reasons:

1. We cannot realise our mission to empower every person and organisation to achieve more without accessibility
2. Accessibility is our path to innovation
3. Our public sector customers are required to procure accessible products

At every major internal Microsoft event I’ve attended this year there has been automatic transcription / captioning of speakers so that deaf or hearing impaired employees can follow along. 5% of the world population (around 360 million people) have some form of hearing difficulty, so the need to use technology to include them in business activities is very real.

However, hearing impediments is not the only area where accessibility in Windows 10 and Office365 is helping:

As I’ve researched more about the various accessibility features I’ve come across some great customer testimonials and case studies about how Windows 10 and Office365 are making a big difference for them on a daily basis. Below is one about Ted Hart who works at Microsoft and was part of the team that improved Skype Translator for English captions/subtitles resulting in deaf people being able to take part in conversations normally:

The next case study is entitled The Power Of Visual Communication showing how Skype video allows students with disabilities to be able to communicate with each other, even when on work experience. The visual nature of Skype means they can use both sign language and also read body language:

Finally, Al Amal School for Deaf Students in the United Arab Emirates shares how the use of tools like Office Mix and video recording in OneNote is proving valuable with their students:

There are numerous blogs from Microsoft that focus on specifically on accessibility  and Microsoft has a dedicated Disability Help Desk that supports video calling with American Sign Language:

Marlee Matlin, the only deaf performer to win an Academy Award for Best Actress, demonstrated how the help desk works:

Translator Tools:

Microsoft have leveraged a number of translator tools that use Machine Learning and the Intelligent Cloud to provide greater accessibility support for all users, particularly those that are hard of hearing:

1. Skype Translator: not only does this do real-time translations between different languages it can be re-purposed to provide an effective transcription of an English to English conversation to support deaf participants.
2. Microsoft Translator phone app: Similar in functionality to Skype Translator, this app for your phone allows you to do transcriptions and translations (over 60 languages) and you can do multi-person chats on the same device or with up to 100 participants by sharing a conversation code.
3. Presentation Translator: A project from the Microsoft Garage, this looks to be a plugin for PowerPoint that will provide real-time captioning on the PowerPoint itself from the speech of the presenter. This is not available yet but is coming soon in a beta trial.
4. Skype Broadcast: The premier Skype meeting tool (available in O365 E5 plans), this will provide real time transcription of the Skype meeting so all participants can follow along.

Other Tools:

Outside of the straight translation tools above, Microsoft are building accessibility into a range of other products that are available now to customers:

1. Video Indexer: This was formerly know as Video Breakdown in the Azure Media Analytics Suite and is currently in free trial. This tool allows you to automatically transcribe speech in a video, OCR scan text contained within the video, provide facial recognition and then index and search across all this content. It’s incredibly powerful.
2. Microsoft Stream: Announced only today from Microsoft as being Generally Available (GA), this is built right into Office365 subscriptions and is a video library tool that also offers speech-to-text, facial recognition and searchable indexes. With granular sharing permissions this is a powerful tool.
3. Accessibility Checker: Build directly into Office365, this tool scans your documents and identifies ways you can make them more user friendly for all users, but specifically those that may fact accessibility challenges. There is no need for a third party plugin to achieve this – it’s baked in by default!
4. Learning Tools / Immersive Reader: Hugely popular in education already, this started as an extra plugin for OneNote Desktop, before being built in directly and also supported in the Web/Online versions in a browser as well. This tool will read text back to the user, highlight adjectives/nouns/verbs and provide coloured overlays to assist dyslexic users.
5. Office Accessibility Center: The one stop shop for Microsoft accessibility content and ideas.
6. Surface Hub: When it comes to providing the best hardware / software combination for collaboration for deaf customers, I think the Hub the best choice given the fully integrated camera experience will provide the ability to use sign language to communicate easily and effectively with remote users. Two videos below show the ease of meetings in with Surface Hub:

It is not just Surface Hub that is helping deaf students, the Surface Pro was highlighted in a video showing how deaf students playing American Football used it to communicate and develop game strategy:

Accessibility examples from Windows 10

As you can see, there is a huge amount of work being done to support all users to make Windows and Office365 a totally accessible product. One small thing I really like is the ability to replace audio cues with visual or text notifications in Windows, an invaluable addition to the user experience for a deaf person.

If you think I’ve missed something or have other suggestions feel free to drop a note in the comments below.