Global Accessibility Awareness Day 2018 is fast approaching on May 17th and will the be the seventh year this day is celebrated. The focus is on an audience encouraging the design of technology (both software and hardware) that will be more accessible to every end user, including those with disabilities.
To support this day, Microsoft are running some cool webinars show casing how Windows 10 and Office365 in Education focuses on creating inclusive classrooms. The three sessions are:
Improving Literacy In Your Classroom With Learning Tools
In this session Mike Tholfsen will show how the Learning Tools in Office365 have demonstrated ability to improve the reading and writing levels of students. These tools are expanding to the Edge browser and Outlook so there is an ever wider audience to benefit from them.
Exploring The Accessibility Features In Windows 10 To Enhance Productivity
I am always surprised how few people know about the amazing built in features in Windows 10 that support all users through accessibility tools such as Narrator, Eye Control, Read Aloud and the recently improved Dictation.
No plugins required – built in from the ground up in Windows 10.
Office365 can now check your content to provide feedback on how accessible it is for all consumers, providing tips on what can be improved. Some of the more geeky/cool features include the ability to embed closed captions in PowerPoint and Sway or using the Presentation Translator to create live, on the fly sub titles to your presentation for those that may be hearing impaired or speakers of other languages (yes, real time translation is supported!).
If you’re looking for ways to support users in your classroom, be they students or teachers, then consider supporting Global Accessibility Awareness Day and sign up for the webinars above.
Microsoft technology in use to help students who are deaf or hard of hearing at Rochester Institute of Technology in Rochester, New York. Pictured pre-Baccalaureate in Science student, Joseph Adjei, from Ghana. (John Brecher for Microsoft)
I read a great case study this morning out of New York City’s Rochester Institute of Technology which has 1500 dear of hearing impaired students, and are now using Microsoft Translator to provide real time captioning and translation to support the American Sign Language (ASL) translators in the lectures.
I’ve blogged about this before here and here and also wrote an article about the impact of Artificial Intelligence (AI) more generally in the classrooms for the Interface Magazine that you can read here.
This case study is really timely as I was in a school in Wellington yesterday working with a teacher who has a profoundly deaf student in her class and having the ability to use something like Translator will certainly add another layer of information for the student in the absence of a full time sign language translator.
Joseph Adjei, a first-year deaf student from Ghana loves Microsoft Translator
In my experience, the Microsoft Translator works best if the presenter/speaker is wearing a mic close to their mouth for the most accurate detection of their speech. Using the default in-built mic on a laptop generally has too much ambient noise and can reduce the accuracy and quality of the transcription and consequently the translation as well if this feature is being used.
If you’re using this in your classes, I’d love to hear how the experience is going so drop a note in the comments below.
“Clippy” was once considered the height of AI assistance!
It is an exercise in stating the obvious to say we are living in a rapidly changing world, where technology is both one of the most disruptive and exciting influences on our society. Yet change is constant, and something that we have experienced forever – in the 17th century King Henry IV of France wished for all his people to have “a chicken in every pot.” Fast forward 300 years to 1977 and Bill Gates’ vision was for a “computer on every desk and in every home” and now with the advent of Artificial Intelligence (AI) and the Internet of Things (IoT) it seems there is a computer in every pot and chicken!
Education is not immune to the increasing influences of technology and yet after a decade working in schools and the wider education sector, I’ve never been more convinced that teachers are the most valuable resource a school can possess and the old scare-mongering that robots will replace them could not be further from the truth. That said, technology and AI is going to empower and enable schools and teachers to do more than ever before and this is at the heart of Microsoft’s vision and is evident through increasingly smart applications designed to help educators and students alike.
Classroom Level Apps:
Familiar Office applications are now ‘super charged’ by the power of the intelligent cloud, utilising Machine Learning (ML) to infuse AI driven features into the products and help teachers improve learning outcomes. This is perhaps most evident in the area of accessibility, where Microsoft works to ensure every student has access to technology in a way that will help them learn.
Presentation Translator is a free plugin for PowerPoint that creates real-time subtitles of what the teacher is saying, displaying them below the presentation. Furthermore, using Azure Cognitive Services, AI-powered speech recognition and translation allows students to hear or read what is being said in their own native language.
The power of Azure’s intelligent cloud enabling Presentation Translator to work
Even in classes where English is the only language being spoken, deaf or hard-of-hearing students can follow along with the real-time transcriptions, either on the teacher’s display or by joining the conversation on their smart phone. From the free app, students can translate the conversation into over 30 languages and ask questions via voice or keyboard entry, which are optionally displayed alongside the teacher’s sub-titles. Presentation Translator maximises and reinforces key learning messages by presenting concepts both aurally and visually to students, as well as providing a searchable typed transcript for revision later.
For the visually impaired, the free Seeing AI app also leverages the massive computing power of the Azure cloud to narrate the world aloud to a student, simply by taking a photo with their smartphone. Seeing AI delivers the relatively simple “read aloud” of text held in front of a smartphone camera, through to aiding a student by scanning currency when paying by cash and reading barcodes off products (with audible beeps to help guide and align the camera). Developed largely by a blind employee at Microsoft, the app can even describe a scene if a photo is taken such as “I think it is a man jumping in the air doing a trick on a skateboard.” For visually impaired students, having a scene in a classroom or playground independently described to them is a liberating experience increasingly possible through the power of AI.
It seems strange to suggest that technology alone can generate a greater sense of inclusion for students. Yet, just like Presentation Translator and the Seeing AI app can draw students into a conversation, AI powered robotics can do the same for students who experience long term medical absences from their classmates. Avatarion is a Swiss company that builds robots connected to the Microsoft Azure IoT Hub that provide absent children a physical presence in class, with full video and audio connections to their hospital or home so they can continue to participate in their learning. The child uses a tablet to control the robot’s movements, speech, send images to classmates and answer questions by raising the robot’s hand and speaking through a connected microphone and speaker. Cloud based AI monitors the robot’s performance at all times, sending valuable information back to the developers to ensure it performs securely and consistently as it represents that ill child in the classroom.
Another combination of AI powered apps that are making significant improvement to literacy levels for students: Microsoft Office Lens and the Immersive Reader in OneNote and Word. Office Lens is a free smartphone app that allows students or teachers to take a photo of text and send it to OneNote or Word where Optical Character Recognition (OCR) is performed by the Azure cloud meaning the text can now be highlighted, have increased spacing between words and syllables and read aloud using the Immersive Reader build right into core Office365 products. Independent research carried out in 2017 shows that the use of Immersive Reader and Learning Tools can significantly improve the reading and writing levels of students and increase equity to educational resources for all students.
School Wide AI Insights:
As the simple-to-use apps above show, Microsoft does not expect every teacher to be a data scientist and AI is increasingly delivered directly via apps that can be used easily by anyone. However, there is also a rich set of tools in the Azure cloud that developers can leverage to build AI powered applications for schools to harness. Tools such as the Office Graph API, Cognitive Services and Media Analytics can be combined with ML to provide deep analytical insights into student performances and then be visually displayed using Microsoft PowerBI dashboards.
The Tacoma Public School District utilised these for AI powered analytics and improved their student graduation rates from 55% to 82.6% over the course of six years. This was not, however, an exercise in technology in isolation. Instead, the school set out to ‘measure the whole child’ and based the programme on four goals:
Helping students achieve academic excellence;
Creating partnerships between parents, community, and staff in educating children;
Focusing on early assessment and intervention to ensure academic success; and
Creating and maintaining safe learning environments.
By pooling all available institutional data on the students in the Azure cloud, along with accessing additional data from government departments and even social media, the Tacoma Public School District was able to use ML and AI to gain deep insights into their students and identify in real time any ‘at risk’ students who required immediate intervention and support.
Closer to home, the Catholic Education of Western Australian (CEWA) have implemented ‘LEADing Lights’, a cloud-first approach to delivering personalised and AI-informed education to the 78,000 students across 163 geographically dispersed schools. Like Tacoma, they are collecting data points from a range of sources and providing rich insights for teachers to take early intervention with students. Dr Cathy Cavanaugh, the Head of Digital Transformation Learning at CEWA, explains how this can look at a classroom level:
“Microsoft Word is used all the time to create essays. Part of the AI in the back end means that as students are writing reports the text can be analysed and sentiment analysed to inform the teacher. There is AI that reads through the text and identifies – based on data points and markers – the level of literacy for the student”
Conclusion:
There are many factors that influence the development and adoption of technology and AI. Brad Smith, President and CLO of Microsoft, recently wrote a blog showing the cultural changes that led to horses eventually being replaced as the primary means of transportation in New York City. Individual students, teachers, schools and even governmental departments will embrace technology at different rates as they perceive the value of it for their communities.
The respective pros and cons of technology in education have long been debated, however I believe Artificial Intelligence provides the key to a step change for the industry. It is not merely ‘digitisation’ of existing resources or teaching practices, instead it is adding a new layer of value by delivering data driven insights and tools to enable access to learning that simply could not be achieved without the power of the intelligent cloud. As this happens, more students will be enabled and empowered to learn, who in turn may go on to be the creators of even smarter tools to help educate the students of tomorrow.
Technology can break down barriers in many different ways and I’ve blogged previously about accessibility in Office365 as well as the amazing Seeing AI App that helps the blind to “see.” When it was first announced, Microsoft Translator was a Garage project, meaning it was something of a passion project for Microsoft developers and was not as yet, not a mainstream product.
This has changed with a big push on how Microsoft Translator can help break down language barriers and create closer connections between school communities, be that the Teacher/Student relationship or the Principal/Parent meeting scenario. In this blog post from earlier today, Chinook Middle School shared how they’re using the Microsoft Translator tool to connect with their linguistically diverse parent community:
This type of technology is tremendously exciting because, in my mind, it allows students who might otherwise miss key ideas or concepts to have a greater chance of learning because they are either hearing or seeing content in their own native language, or are seeing transcribed subtitles of what the speaker is presenting.
I’ve used Microsoft Translator in a number of presentations to simply add English subtitles to assist those with hearing difficulties or English Speakers of Other Languages (ESOL) to see in typed text what it is that I am saying (since I have a tendency to speak quite fast at times. The accuracy of transcription is remarkably high, considering it is in real time and having to deal with the vagaries of the New Zealand accent!
There is now a dedicated page for Microsoft Translator with an Education focus and I encourage you to check out the resources included on it:
One of the really great features of Translator is that it can be trained to learn technical words as well. By including these in the notes section of a PowerPoint slide deck, Translator will more effectively interpret these words when they are being spoken by the presenter.
To learn how to get started with Microsoft Translator check out this video:
My Point of View:
Teaching is changing in often quite major ways, from an increasingly ethnic/linguistically diverse student audience through to building redesigns with an emphasis on larger open and flexible spaces. Both of these two scenarios would benefit from the presenter or teacher using Microsoft Translator because the use of translation and even English subtitles will enable the audience to follow along even if they can not hear the presenter clearly, they can see the subtitles on the presentation.
I wrote in my last blog post that literacy strategies designed to support dyslexic students are equally effective with the majority of students and having subtitles will assist all students as well – not just those that are hard of hearing.
Most schools in New Zealand have an International Department where students are being supported in developing their English skills and having the ability to do real time translation could be simply another tool that can be exercised to assist these students in being understood. Similarly, I’ve run many Mystery Skype sessions but have been limited to other English speaking schools. Imagine the levels of excitement amongst students if they were able to talk with students from other languages and be easily understood. This would definitely expand the list of time zones and schools that could connect with each other.
As technology plays an increasingly disruptive role in society, I love it when tools like this emerge that harness the massive power of Machine Learning and channel it into an incredibly positive product like Translator that will connect more people than ever before.
Lastly, for those more technically inclined readers, have a look at the technology behind the scenes to get this running:
It’s a great read and I encourage you to link through to this and see it in detail. It includes the following video case study:
In my previous work as a secondary school social sciences teacher, we were often focusing on lifting the learning outcomes for targeted students. Literacy levels were something that was important for all, and often implementing strategies for improving educational outcomes for Māori and Pasifika students was also a focus. The point here was that the messaging around targeted strategies for these groups was that it would also help improve outcomes for all students because it was fundamentally sound pedagogy.
The same is true for the accessibility features in Office 365 and Windows 10 – ensuring compliance with accessibility standards will help all students (and indeed, all users, such as teachers, administration staff and parents). The blog post from the MSAU team highlighted the following key features in the area of accessibility:
Check Accessibility
In the Review Tab in Office 365, simply click on the Check Accessibility button to see if your document – test, assignment, teaching notes – can be read aloud. The key is to use styles and avoid using the return key to create space on the page, which can be done in the paragraph styles. That way when your document is being read it can alert the reader – “Heading: Year 5 test”. “Subheading: Answer any two questions”. “Subheading. Question One”. You can immediately grasp the difference this makes to vision impaired students or those who respond better to the spoken word. Plus, by applying styles, you’ll be learning some good writing habits yourself!
Learning Tools
Learning Tools gives students new ways to approach learning tasks in Word, OneNote, Outlook, Office Lens or ePubs. The Immersive Reader is a standout. It enables students to have a text read to them, giving vision-impaired students learning independence and putting them on an equal footing with their peers. The Dictation tool allows students for whom writing is an impossibility to record their thoughts without writing. And the contrast tool is a powerful decoding aid for dyslexic students. Learning tools don’t just make a huge difference to students with learning difficulties, they can increase fluency for English language learners and help emerging readers to progress to higher levels.
Office Lens
This app is a game changer. A free download, it enables students to snap a photo of, for example, the class whiteboard, a printed page or rough sketch on paper. They can then import it into OneNote, OneDrive, Word, PowerPoint, Outlook or Immersive Reader where it appears as editable text. Just from a research perspective it’s a huge time-saver, enabling every student to collate information quickly without having to rekey. For students with reading difficulties it means that text can then be enlarged or different fonts or colour backgrounds to make it more accessible. Now even the school canteen pricelist can be quickly scanned and read back to a blind student. Plus, teachers can save a lot of time after team brainstorming meetings by simply photographing the whiteboard ready to take the next step – from sharing to editing!
Text from a photo on the left is scanned via OCR and read in Immersive Reader
Presentation Translator
Presentation Translator is a new add-in to PowerPoint that translates and subtitles live presentations, displaying subtitles directly on a presentation in any one of more than 60 languages. By unmuting the microphone, teachers can also allow students to ask questions by typing or speaking, which are displayed for all to see. This enables hearing impaired students to follow along with the class on their phone, tablet or computer and participate in the discussion without requiring assistance.
Since writing this blog I see that the Microsoft Garage have also released a new product called MS Dictate which is a plugin for Outlook, Word and Powerpoint that allows you to dictate text using the same speech-to-text engine used by Cortana. You can download it for free here.
Recently I’ve been working with a partner that has a school for deaf and hearing impaired students as a customer. It’s been really interesting exploring how technology is used in environments like this, where the need for video communication to enable sign language is paramount.
As a result, I’ve been digging into the accessibility options within Microsoft’s Windows 10 and Office365 products and it has reinforced the observations I’ve noticed already since joining Microsoft at the start of the year: accessibility and inclusive design really is at the heart of all Microsoft products.
Accessibility is a priority for Microsoft for three key reasons:
We cannot realise our mission to empower every person and organisation to achieve more without accessibility
Accessibility is our path to innovation
Our public sector customers are required to procure accessible products
At every major internal Microsoft event I’ve attended this year there has been automatic transcription / captioning of speakers so that deaf or hearing impaired employees can follow along. 5% of the world population (around 360 million people) have some form of hearing difficulty, so the need to use technology to include them in business activities is very real.
However, hearing impediments is not the only area where accessibility in Windows 10 and Office365 is helping:
As I’ve researched more about the various accessibility features I’ve come across some great customer testimonials and case studies about how Windows 10 and Office365 are making a big difference for them on a daily basis. Below is one about Ted Hart who works at Microsoft and was part of the team that improved Skype Translator for English captions/subtitles resulting in deaf people being able to take part in conversations normally:
The next case study is entitled The Power Of Visual Communication showing how Skype video allows students with disabilities to be able to communicate with each other, even when on work experience. The visual nature of Skype means they can use both sign language and also read body language:
Finally, Al Amal School for Deaf Students in the United Arab Emirates shares how the use of tools like Office Mix and video recording in OneNote is proving valuable with their students:
Marlee Matlin, the only deaf performer to win an Academy Award for Best Actress, demonstrated how the help desk works:
Translator Tools:
Microsoft have leveraged a number of translator tools that use Machine Learning and the Intelligent Cloud to provide greater accessibility support for all users, particularly those that are hard of hearing:
Skype Translator: not only does this do real-time translations between different languages it can be re-purposed to provide an effective transcription of an English to English conversation to support deaf participants.
Microsoft Translator phone app: Similar in functionality to Skype Translator, this app for your phone allows you to do transcriptions and translations (over 60 languages) and you can do multi-person chats on the same device or with up to 100 participants by sharing a conversation code.
Presentation Translator: A project from the Microsoft Garage, this looks to be a plugin for PowerPoint that will provide real-time captioning on the PowerPoint itself from the speech of the presenter. This is not available yet but is coming soon in a beta trial.
Skype Broadcast: The premier Skype meeting tool (available in O365 E5 plans), this will provide real time transcription of the Skype meeting so all participants can follow along.
Other Tools:
Outside of the straight translation tools above, Microsoft are building accessibility into a range of other products that are available now to customers:
Video Indexer: This was formerly know as Video Breakdown in the Azure Media Analytics Suite and is currently in free trial. This tool allows you to automatically transcribe speech in a video, OCR scan text contained within the video, provide facial recognition and then index and search across all this content. It’s incredibly powerful.
Microsoft Stream: Announced only today from Microsoft as being Generally Available (GA), this is built right into Office365 subscriptions and is a video library tool that also offers speech-to-text, facial recognition and searchable indexes. With granular sharing permissions this is a powerful tool.
Accessibility Checker: Build directly into Office365, this tool scans your documents and identifies ways you can make them more user friendly for all users, but specifically those that may fact accessibility challenges. There is no need for a third party plugin to achieve this – it’s baked in by default!
Learning Tools / Immersive Reader: Hugely popular in education already, this started as an extra plugin for OneNote Desktop, before being built in directly and also supported in the Web/Online versions in a browser as well. This tool will read text back to the user, highlight adjectives/nouns/verbs and provide coloured overlays to assist dyslexic users.
Surface Hub: When it comes to providing the best hardware / software combination for collaboration for deaf customers, I think the Hub the best choice given the fully integrated camera experience will provide the ability to use sign language to communicate easily and effectively with remote users. Two videos below show the ease of meetings in with Surface Hub:
It is not just Surface Hub that is helping deaf students, the Surface Pro was highlighted in a video showing how deaf students playing American Football used it to communicate and develop game strategy:
Accessibility examples from Windows 10
As you can see, there is a huge amount of work being done to support all users to make Windows and Office365 a totally accessible product. One small thing I really like is the ability to replace audio cues with visual or text notifications in Windows, an invaluable addition to the user experience for a deaf person.
If you think I’ve missed something or have other suggestions feel free to drop a note in the comments below.
It seems strange to suggest that technology alone can generate a greater sense of inclusion for students. Yet, just like Presentation Translator and the Seeing AI app can draw students into a conversation, AI powered robotics can do the same for students who experience long term medical absences from their classmates. Avatarion is a Swiss company that builds 
This type of technology is tremendously exciting because, in my mind, it allows students who might otherwise miss key ideas or concepts to have a greater chance of learning because they are either hearing or seeing content in their own native language, or are seeing transcribed subtitles of what the speaker is presenting.
I’ve blogged previously about my observation that accessibility is at the heart of product design
