iPhone Magnifier and Voice over

The YouTube channel by Kristy Viers has been helpful in understanding how they use an iPhone and other devices. Here’s a video on VoiceOver in the Camera app: [VIDEO]

It describes the scene, tells you how many faces are visible, whether or not the image is focused, lighting conditions. [VIDEO] They can also generate alt text for photos which do not have that information already. Here's a video of the person in the original article describing this feature: [LINK]

Thanks for the link, super interesting to watch. Over the years as a programmer I've picked up knowledge about web accessibility from a mechanical, standards/implementation focused point of view, but this video is helping me realize that my understanding has been lacking a level of depth and empathy for the real people using these tools. Looking through some of her other videos I found this demonstration of camera-to-speech in the iPhone pretty awesome–I had no idea that this was a feature: [VIDEO]

This [VIDEO] is a recent thing I found mind blowing; 2020 iPhone can do on-device image processing and object recognition and speak a description of what the camera sees. Just quietly a builtin feature. 2010 iPhone, I was mind-blown by the Word Lens app which could do OCR on text in the camera feed, translate the words into another language, and overlay the results on the image in near-realtime. [LINK] These are tasks I had never seen any computer do in any circumstances, things which would be unthinkable on a Java MIDP Blackberry from 2005, or a Pentium II desktop from 1997. Compare this to the wearable augmented reality devices Professor Steve Mann was building through the 80s and 90s[1] and this "describing the image" is so so far ahead in so many ways - processing power, imaging quality, battery life, storage space, size, weight, convenience, reliability, it's just mindblowingly better, neural networks and fast chips and solid state storage is a step change in a way that "faster" isn't enough to really convey. Google tells me "Derivatives have a long history in the United States, dating back to the founding of the Chicago Board of Trade in 1848.", you're telling me people are doing that but with blockchain, why is that interesting at all, why mind blowing? [1] [LINK]

I'll one-up this a bit: a blind woman called Kristy Viers demonstrates on YouTube how she uses various things, among them iPhone with the onscreen six-finger Braille keyboard. But also she uses the ‘magnifier’ app to tell her what is there around her, i.e. what the camera sees: [VIDEO] (The videos are filmed by her boyfriend, in case someone has to wonder.) I realize that the vid is not too informative to a blind person, other than a vague demo. IDK where to find more info on the setup, since I'm not a user of iPhones myself. Comments on YouTube say that the magnifier app is in the accessibility options, that it adds a home screen app when enabled, and that it works at least on iPhone 10 since that's what Kristy uses. I would also guess that such usage requires persistent mobile connection and consumes considerable traffic and battery—but again, with modern tech magic I won't bet against it working locally on the phone. On Android, the feature called ‘Lens’ in the Google's camera app can describe some things, more useful to seeing people IMO: e.g. the model of a car—basically the same as Google's image search does, these days. Also purportedly can read and translate text signs. But IDK how far that goes, never heard of it doing stuff like those descriptions by the iPhone magnifier.