Microsoft announced the expansion of Copilot Vision with Highlights to Windows on Wednesday. Initially, the new artificial intelligence (AI) feature will only be available in the US, but the company plans to bring it to other regions soon. It uses computer vision and tool use to analyse the content on the user’s screen and assist them with queries. Copilot Vision was first unveiled by the Redmond-based tech giant in December 2024, and in April, it was integrated within the Microsoft Edge browser.
Copilot Vision on Windows Gets a New Highlights Feature
In a blog post, the tech giant announced that Copilot Vision is finally being rolled out to Windows devices in the US. It will be available for both Windows 10 and Windows 11. The company said the feature will be introduced in more non-European countries soon. Notably, Copilot Vision on Windows is part of Copilot Labs.
Copilot Vision on Windows supports real-time voice, letting users ask verbal queries about the content. This version of the feature supports up to two apps simultaneously and can draw context from both screens at the same time. This means users can ask more complex questions, such as, “Which image from my gallery goes well with the story I’ve written in Word?”
The new Highlights feature in this version will allow users to command Copilot to show certain settings and other assistive help. It can tell users where to click and how to navigate to an option. Sharing some examples of the use cases of the feature, Microsoft said users can ask Copilot to give tips while playing a game, ask it for help to improve the lighting in a photo, or to review a travel itinerary.
To use Copilot Vision on Windows, users will have to open the Copilot app and click the new glasses icon in the composer. Then, users can select the browser window or app they want to share with the AI chatbot, and it can start assisting them. To stop sharing the app or browser window, users can press the X button in the composer. The company highlighted that the feature is entirely opt-in.
