Google is transforming the way we interact with our surroundings, elevating its visual search app, Google Lens, with a game-changing feature: the ability to ask questions about videos. This new capability, currently available for English-speaking users on Android and iOS, enables you to record a video, identify objects of interest, and get answers almost instantly.
Powered by Google’s advanced AI, the feature leverages a specialized version of the Gemini model, which has already been integrated into several of Google’s top-tier products. According to Lou Wang, Director of Product Management for Google Lens, this customized Gemini model helps Lens understand the video content and provide insightful answers.
Wang explained the new feature during a recent press briefing: “For instance, if you come across a fascinating school of fish while recording, you can ask, ‘Why are they swimming in circles?’ Lens will generate an overview explaining their behavior and even link to additional resources for further learning.”
How to Access Google Lens’ Video Feature
To unlock this cutting-edge video analysis capability, users must join Google’s Search Labs program and opt into its experimental “AI Overviews and more” features. Once enrolled, accessing Lens’ video mode is as simple as holding down the shutter button in the Google app. As you capture a video, you can pose questions about any object in view, and Lens will tap into its AI Overviews feature to provide contextually relevant answers.
This AI-powered approach is designed to enhance the user experience by focusing on the most “interesting” or significant frames in the video. Google Lens extracts key information from those frames, helping ground its answers in the context of the visual content, making the responses more precise and useful.
“All of this stems from the way users are already engaging with Lens,” Wang noted. “By removing the friction of asking questions in the moment and satisfying people’s curiosity, we expect this feature to naturally resonate with users.”
A Competitive Edge in Real-Time AI Video Analysis
This innovative feature for Google Lens arrives at a time when major tech players are racing to integrate AI-powered video analysis into their products. Meta recently previewed a similar tool for its Ray-Ban Meta AR glasses, which will allow users to ask real-time questions about their surroundings (e.g., “What kind of flower is this?”). OpenAI has also teased a feature for its premium ChatGPT tool, Advanced Voice Mode, which will eventually analyze videos in real time.
However, Google appears to have taken the lead in this space — at least for now. Lens’ video analysis may not be fully real-time like its competitors, but it offers a practical and accessible step forward. That said, there are still questions about how well the feature performs in real-world settings, as no live demos were shown during the announcement. Google’s past struggles with overpromising AI features add a layer of caution to its unveiling.
More Than Just Video: Enhanced Visual Search Capabilities
In addition to its video analysis feature, Google Lens is also boosting its existing capabilities, offering even more ways to search with a combination of images and text. For example, users can now take a picture in the Google app, then immediately ask a question about it by speaking. This feature is accessible to all English-speaking users, not just those enrolled in Search Labs.
And it doesn’t stop there. Google Lens is ramping up its e-commerce functionality. When Lens recognizes a product — whether it’s in a photo you upload or snap in real time — it will display a variety of relevant details, including pricing, available deals, brand information, and reviews. This feature is currently limited to certain countries and shopping categories, such as electronics, toys, and beauty products.
For instance, if you spot a stylish backpack while out and about, you can snap a quick photo with Lens, and it’ll identify the product, offering instant access to key details like where to buy it, how much it costs, and even customer reviews.
A New Era of Shopping with Lens
The enhanced product identification feature in Google Lens opens up a significant opportunity for Google to further monetize its search app. Shopping-related queries already account for an impressive 4 billion Lens searches each month. Given that shopping is such a major part of the Lens experience, it’s no surprise that Google is also weaving advertising into these search results.
The product results page will now display relevant ads alongside details about the identified item, allowing users to quickly compare prices, see options, and make more informed purchase decisions. Google, whose core business revolves around advertising, is clearly capitalizing on this trend.
Final Thoughts
Google Lens’ new ability to answer questions about videos is a major leap forward in the world of AI-powered visual search. By using a customized AI model to break down complex video content and offer timely, informative answers, Google is pushing the boundaries of what its visual search app can do.
Whether you’re curious about the world around you or simply looking for the best deal on a new gadget, Lens is steadily becoming an indispensable tool for learning and shopping. While it remains to be seen how well the video feature performs in real-world conditions, the potential for this technology is vast. In the fast-evolving AI landscape, Google’s latest updates to Lens are a bold step forward in making digital interaction with the physical world more seamless and intuitive.
