Microsoft's Copilot Vision raises privacy concerns by reading your screen

On December 5, 2024, Microsoft launched a preview of its new AI tool, Copilot Vision, exclusively for users in the United States. This innovative feature is integrated into Microsoft Edge and is a part of the Copilot Labs program, which allows users to engage with experimental AI capabilities by subscribing to the Copilot Pro plan. The tool is designed to analyze both text and images on websites, enabling users to ask questions and receive contextual responses about the content they are viewing. The service aims to enhance user experience by providing assistance while browsing, allowing for inquiries such as extracting a recipe from a cooking website. To use Copilot Vision, users must enable it within their Edge browser. Once activated, the AI reads along with users as they navigate the web, offering concise answers or summaries, and even translating text when necessary. Furthermore, it can assist in various capacities, such as identifying discounts in online store catalogs or acting as a helper during online games like chess. This interactive experience is designed to make the web more accessible and engaging, as the AI learns to respond to user questions in real time. Microsoft has taken care to prioritize user privacy with Copilot Vision, emphasizing that any data processed during a session is deleted immediately afterwards. This approach is likely a response to previous concerns regarding privacy and data management within AI applications. The source of processed information, including images, audio, or text, will not be retained or utilized for training future AI models in this initial preview stage. This insistence on data privacy is critical in light of past controversies surrounding AI and privacy, which have left companies like Microsoft wary of public reception. At this preliminary stage, however, Copilot Vision's capabilities are limited to a specific list of approved websites, as Microsoft aims to ensure compliance with legal standards and publishers' concerns. High-profile cases, such as a lawsuit from The New York Times, have pushed the tech giant to respect web content restrictions more closely. This selective access means that users won’t be able to query content from paywalled or sensitive sites for the time being, as the company cautiously expands the feature in response to both user feedback and legal obligations. Microsoft has stated that they will continually evaluate and add to the list of accessible websites to enrich the user experience over time.