An ‘Enhanced Visual Search’ toggle in the iOS Settings lets you search for important landmarks in the Photos app, but it needs to share data with Apple to do so.
Apple has long marketed itself as a champion of user privacy, building its brand on features that protect users from prying eyes. But every now and then, decisions surface that seem to chip away at this privacy-forward reputation. A recent discovery by developer Jeff Johnson is raising eyebrows: a feature in the Apple Photos app called “Enhanced Visual Search” that appears to share certain photo data with Apple by default.
Here’s what you need to know and why it matters.
The Feature in Question
Enhanced Visual Search is designed to expand the functionality of Apple’s Visual Look Up, a tool introduced in iOS 15. Visual Look Up allows you to identify elements in your photos, such as plants, animals, and even laundry symbols. Enhanced Visual Search takes it a step further by letting you identify landmarks in your photos and search for them using names. For instance, if you’ve snapped a picture of a famous building, you can swipe up on the image, select “Look Up Landmark,” and see a card with its name and information.
On the surface, this seems like a helpful, convenient feature for travelers or those curious about their photo collections. However, the underlying mechanism of how it works and the fact that it’s enabled by default without clear communication has raised some concerns.
How to Check if It’s Enabled
Curious if this feature is on for your device? Here’s how to check:
- On an iPhone: Open the Settings app, navigate to Photos, and look for the “Enhanced Visual Search” toggle.
- On a Mac: Open the Photos app, go to Settings, and find the same option.
When I checked my iPhone 15 Pro, the toggle was already switched on by default. Many users might not even realize this feature exists, let alone that their photos data might be shared with Apple.
What’s Happening Behind the Scenes
According to Apple’s machine-learning research blog, Enhanced Visual Search starts with an on-device machine-learning (ML) model that analyzes a photo for a “region of interest” (ROI). If the model detects an ROI, such as a potential landmark, it calculates something called a “vector embedding” for that region.
What is a vector embedding? In simple terms, it’s a mathematical representation of data. IBM describes embeddings as a way to transform a data point (like an image or a word) into an array of numbers that capture its characteristics. These embeddings are encrypted and then sent to Apple to be compared against a global database of landmarks.
Apple claims that the process is privacy-preserving. The data sent to its servers is a condensed, encrypted form of the image data, legible only to machine-learning models. While this approach appears to protect users from overt privacy risks, it still involves sharing information with Apple without explicit opt-in consent.
Why This Matters
The issue here isn’t necessarily about whether Apple is misusing the data; it’s about transparency and choice. Unlike sharing Siri recordings or analytics data—which requires users to opt in—this feature is enabled by default. Users must proactively discover the toggle and turn it off if they don’t want their data shared. For a company that prides itself on putting users in control of their privacy, this feels like a misstep.
A Case for Opt-In Settings
Johnson and others argue that this feature should follow an opt-in model, similar to other Apple features that involve sharing data. This approach would align better with Apple’s privacy-focused ethos and ensure users are fully informed about what’s happening with their data.
While Apple has gone to great lengths to keep the shared data private—from encrypting the embeddings to ensuring they’re only used for matching—the decision to make the feature opt-out rather than opt-in feels out of step with the company’s usual commitment to transparency.
How to Protect Your Data
If you’re uncomfortable with the idea of sharing photo data, here’s how to disable Enhanced Visual Search:
- Open the Settings app on your iPhone.
- Navigate to Photos.
- Toggle off Enhanced Visual Search.
On a Mac, the steps are similar: go to the Photos app’s settings and turn off the feature.
The Bigger Picture
Enhanced Visual Search highlights a broader challenge in balancing innovation and privacy. Features like these make technology smarter and more convenient, but they also require clear communication and user consent. Apple’s privacy reputation has been a cornerstone of its brand, and decisions like this risk eroding trust.
Ultimately, the debate isn’t just about Apple. It’s about how tech companies handle user data in a way that respects privacy while still delivering cutting-edge features. Enhanced Visual Search may be a small piece of the puzzle, but it serves as a reminder to all of us to stay vigilant and informed about how our data is being used.
