Enthusiasts have studied the APK of the upcoming version of Google Search 10.61 and discovered a new feature there. It is probably based on the same AI module as the old What's on my Screen option. The new feature is called Smart Screenshots and expands the ability to work with screenshots.
When you take a screenshot, the familiar control panel opens, but with a new Lens button. Behind it is a tool for optical character and visual recognition, which allows you to take arbitrary fragments of a screenshot and use them in a search query. This can be useful in a situation where the user simply cannot formulate what problem he is facing. It remains to take and show it to the search engine in order to find a solution.
From the information available, it is not entirely clear how the data collected by Lens will be used. Can they be used to search through other systems, or is it strictly about supporting internal Google services? Previous options like editing and sharing remain unchanged.