Reflections on Microsoft’s AI Tour In Sydney

We’re bringing together decision makers, developers, and industry experts to unlock the potential of AI. Don’t miss this opportunity to connect with thought leaders and explore new paths forward.

Microsoft AI Tour Sydney Homepage

This week I attended Microsoft’s AI Tour on the Sydney stop – an impressive event and it really felt like a throwback to Microsoft events of old – plenty of cash thrown at the event and a really positive vibe with a heap of sessions to attend and some keynotes from leaders in the business:

TLDR; Too Long, Didn’t Read

If you’re pressed for time, here are the bullet points of the event for me:

• Microsoft is “all in” on AI – no revelation here, but this event reinforced the fact that every aspect of their business is going to be integrating Copilots: from Windows, to D365, M365, Azure Services, Security – everything.
• 14months on from the release of ChatGPT, there has been significant evolution in the capabilities of generative AI offerings from MSFT and OpenAI and over 1billion users have used ChatGPT in one format or another.
• Some Australian educational institutes, notably the Departments of Education in South Australia and New South Wales have released custom LLM and chat bots (EduChat) that is focused on equity of access to GAI with a focus on security and parameters to restrict chats to education focused themes. The University of Sydney is also in a rapid pilot phase with various AI powered services.
• Scott Guthrie, the EVP for Cloud and AI outlined four key areas he believed AI would create new opportunities:
  • Enrich employee experiences
  • Reinvent customer engagements
  • Reshape business processes
  • Bend the curve on innovation
• Microsoft’s messaging on an organisation/customer data:
  • Very clear, specific language: “your data is your data”
  • This is about trust, but it’s also about responsibility. You’re expected to be maintaining strong data hygiene practices and restricting access on a ‘needs to know’ basis.
  • I suspect this is to some extent around liability: it seems almost inevitable to me that these AI services are going to comb and index organisational data and accidentally expose sensitive data to unauthorised users.
  • I’d go as far as saying many employees will actively try to use AI tools to hunt for sensitive data inside their organisation – if the correct data permissions are not in place there is real risk here for data leaks.
• I’d argue the missing messaging here is the need for organisational AI governance policy
  • Humans are inherently lazy – as they see better and better results, they’ll check the output less and less.
  • I suspect organisations will need to enforce policies and accountability at an individual level for any over reliance on GAI output.
• Lots of data points about the efficiency gains from Copilot. MSFT have been using it internally for their own support services. Some examples:
  • 31% increase in first call resolution;
  • 12% increase in overall customer satisfaction;
  • 14% more cases resolved per hour.
• Copilot is just that – a guide by the side, it is not an Autopilot
  • This is my language of “Copilot not an Autopilot” but it rings true with what is being shared a these sessions – the human element is still required at this stage to be able to discern and verify the accuracy of what is being generated.
  • A case in point – we saw an example where a MSFT marketing manager created a 60 word bio from his LinkedIn profile with Copilot – I thought I’d try the same, pointed Copilot at my exact LinkedIn profile and it still “hallucinated” and got key facts wrong – e.g. said I was studying towards a PhD at Monash University (spoilter: I’m not!)

• Microsoft has done a Work Index Trend for a few years now – unsurprisingly, lots of AI data being shared (and clearly a driver to promote adoption for MSFT):
  • 70% users were more productive with Copilot
  • 68% said Copilot improved the quality of their work
  • 64% spent less time processing email
  • 71% saved time on mundane tasks
  • 75% less time searching for information in their files
  • 77% Copilot users said don’t want to give it up
  • 10hrs+ saved in time per month (best users)
• Copilot Studio is a lowcode offering to extend the capabilities of Copilot. There are over 1200 connectors already built into third party apps and platforms, and you can build your own as well. This looks truly useful if you’re going all in on Copilot
  • Microsoft Copilot Studio | Extend Copilots or Create Your Own
• What are the barriers for AI adoption? According to a panel of MSFT staff:
  • Lack of skilled AI workers
    • Security concerns – where is an organisation’s data being sent to and analysed, what does “trust” look like in a world of AI?
  • Lack of confidence in AI capabilities and accuracy.
• Organizations that have been slow / slack are going to face increased barriers to AI adoption
  • This came from from a partner question actually – not just slack in AI data hygiene but also slow to adopt cloud services and now wanting to use AI with on premise databases etc.
  • Response from MSFT to this observation: it’s definitely easier to adopt AI if you’re in the cloud, but it’s not impossible if you’re on premise still

Slightly Deeper Dive On Specific Sessions I Attended:

The Future of Education In The Era Of AI

• An interesting session that didn’t teach me too much, but a few links and thoughts:
• Assessment reform for the age of artificial intelligence | Tertiary Education Quality and Standards Agency (teqsa.gov.au)
• What to do about assessments if we can’t out-design or out-run AI? – Teaching@Sydney
• How AI Could Save (Not Destroy) Education | Sal Khan | TED – YouTube
• NSW and SA Departments of Education leading the way with custom AI Chat bots
  • heavily safeguarded, multiple filters (swear words in 150 languages for example
• The identified challenges from the presenters around AI in education were:
  • Organizational alignment internally
  • Operational model development
  • Governance and data preparation
  • Addressing equitable access to AI tools
• The session continued with a panel discussion hosted by Adam Pollington (Sales Lead for Microsoft ANZ Education and featured Jim Cook, Innovation Lead at Sydney University, Dan Hart, Data Science Lead at NSW Department of Education and finally Anthony England, Director of Innovation at Pymble Ladies College. Together, they had some interesting things to share with some highlights being:
  • Sydney University is rolling out ‘dozens and dozens’ of small scale PoCs related to AI currently. They opted to lead off with a policy navigator tool with the legal department at the Uni – relying on the belief if legal was on side with this technology, they could scale it further.
  • NSW DoE annouced on the day a large scale pilot of their custom LLM EduChat (details here) – they aim to promote this over ChatGPT and learn what their own model is capable of and where it needs to improve.
  • Anthony England brought a very education focus to the discussion, balancing the role of tech and educators nicely. His goal is to have 60 AI champions across the school, with a great quote of “focus on staff, not tools; process not product”
  • All recognised the challenge of AI “hallucinations” – where incorrect information is surfaced up and how do end users detect/validate and ultimately correct for this.
  • The session finished with Adam asking each panelist what their guidance was for orgs wanting to ramp up AI usage:
    • Jim Cook: Manage the expectations of stakeholders
    • Dan Hart: Focus on user engagement
    • Anthony England: Provide an environment where users can learn, play and fail, and then iterate on top of that.

Keynote: Scott Guthrie, Microsoft EVP for Cloud and AI

• You know Microsoft is not holding back when they send an EVP down to Australia and Scott gave a 15-20mins presentation covering a few key areas of where he sees AI having significant impact:
  • Enrich employee experience
  • Reinvent customer engagement
  • Reshape business processes
  • Bend the curve on innovation
• There were the predictable messages around MSFT runs on trust and that “your data is your data” – which is reassuring and I think MSFT recognise the need for customers and partners to have to trust that their organisational IP is not going to be used to train the underlying LLM
• He then focused on three core areas (I won’t repeat in detail):
  • Unlock productivity across your business with Copilot
  • Build transformational AU solutions with Azure
  • Secure your business end to end with GAI
• Of those, the focus on security at the end was least detailed – it seems like Copilot for Security is still a work in progress.
• This was followed by a fireside chat with the CIO from Commonwealth Bank of Australia who had some interesting insights:
  • If you don’t know where your data is, Copilot will find it. Validate your data strategy, clean it up – good hygiene is critical.
  • they are building a ‘personal banker’ into their banking app that will be powered by AI
  • He then focused on the need for quantifying data when it came to building a business case for recommendations on spending on Copilot licenses: staff in the pilot were asked if they would prefer a $50/m lunch voucher or a $50/m Copilot license – 75% chose the Copilot license! They also analysed the impact of Github Copilot with their developers – 75% of respondents said it was very helpful and after studying their code commits over a 12 week span (2 sprints) they identified that 1/3 of the lines of code being committed (80k lines) were written by Copilot.

Modern Work and the AI Powered Organisation

• This session was mostly disappointing and largely run by marketing (correlation?)
• The key info was the sharing of Work Trend Index data (a form of marketing in itself!)
  • 70% users were more productive with Copilot
  • 68% said copilot improved the quality of their work
  • 64% less time processing email
  • 71% saved time on mundane tasks
  • 75% less time searching for information in their files
  • 77% Copilot users said don’t want to give it up
  • 10hrs+ saved in time per month (best users)

Agile Analytics – A Tour Through the Possible (Enhancing Aure OpenAI with your data, systems and APIs)

• This was arguably the most interesting session and was presented by an Australian based Microsoft partner called Agile Analytics.
• They were more focused on taking OpenAI solutions (running on Azure) and extending them into bespoke products rather than simply onselling Microsoft’s 1st Party Copilot offerings.
• They presented three live demos:
  • Improve Operations – querying your own systems
  • Improving Health – unstructured data to structured
  • Improving Finance – improving processes and democratization of AI
• I did have a bit of an ‘a-ha’ moment from this session: the scenario was a team meeting and the manager noticed ‘Sarah’ was a bit flat. By putting a single query into a bespoke chatbot powered by a custom LLM, he could type ‘Sarah Brown’s HR record’ and it went off via API and came back with a pre-formatted output of her HR record, including her annual leave. In the scenario the manager notes Sarah has not taken any leave for 12months so suggests she takes a few days break. This was positioned as democratizing access to knowledge: through a single interface a manager could ask (in natural language) for info on employees and it would be returned, without the need to go to the HR tool itself, log in, pull the record, check the annual leave etc.
  • I could certainly see scenarios where this could be a good time saver, especially if this is pulling from across a range of systems – e.g. show me the top sellers and their sales summary to date, and then match this with their internal training – any correlation? Are top sellers effective because of training or not?
• Another comment made by the presenter was that AI is an area that really rewards creativity – e.g. he saw one of his team providing a database schema to OpenAI – the chatbot was then trained to look at the users query, look at the provided database schema, then it would automatically generate the SQL query needed to extract the answer from the underlying SQL database – this is clever.
• The healthcare example was presented by a Microsoft employee and had some cool demos.
  • One example was taking a doctor’s shorthand notes and complex medical jargon, copy/pasting into an LLM and having it write the patient’s complete discharge notes in what she described as “universal language” i.e. translating the Doctor’s shorthand and medical terminology into comprehensible notes for the patient to be able to read. I could see this being a real time saver.
• These scenarios were extended into the Financial Services Industry – the following two examples were shown on screen (poor quality images sorry) – on the left is a basic response on whether someone qualifies for lending, on the right a more complex and nuanced reply:

The final slide presented was the recommended decision making tree on when to use first or third party AI solutions which I thought was helpful:

Leaning Into AI to Radically Transform The World Of IT

• This was a panel discussion:
  • Imogen Schifferie chaired it from MSFT.
  • Alistair Speirs – Microsoft Azure Infrastructure
  • Lee Hickin – Microsoft Asia AI Lead
  • Tal Dagan – Atera
• The panellists were asked about blockers to IT adoption:
  • Skilled workers – Tal Dagan
  • Security – concerns around where company IP is being sent and analyzed. AI is so nascent there is a need to rethink what we mean by “trust” – Alistair Spiers
  • Need to build confidence in AI (Security, safety etc) – – we need more skilling on how to use AI safely and effectively – Lee Hicken.
• When asked which industries were the fastest to adopt IT it was clear:
  • IT itself – putting itself at the forefront of tech and adopting it the most quickly
  • Industries that are the most heavily regulated and monitored as they already have significant governance and oversight structures in place: healthcare, education, finance.
• I asked a question at the end of this session around what is required around organisational governance policies and requirements of employees to actually vet/validate that that GAI content is accurate – all felt there was a strong need for individuals to remain vigilant in these early days of AI and not rely 100% on the content created (AI hallucinations was mentioned again)
  • this reiterates my idea: It’s a Copilot and not an Autopilot!