American Family Vision Clinic Logo
Authors
Christina M. Funke; Judy Borowski; Karolina Stosio; Wieland Brendel; Thomas S. A. Wallis; Matthias Bethge

Five points to check when comparing visual perception in humans and machines

publication date
March 2021
Category
see more

Abstract/Introduction

With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed toward comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct, and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect the interpretation of results and that several analytic tools can help to overcome this human reference point. In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks. Thereby, we show that contrary to previous suggestions, feedback mechanisms might not be necessary for the tasks in question. The third case study highlights the importance of aligning experimental conditions. We find that a previously observed difference in object recognition does not hold when adapting the experiment to make conditions more equitable between humans and machines. In presenting a checklist for comparative studies of visual reasoning in humans and machines, we hope to highlight how to overcome potential pitfalls in design and inference.


Conclusion/Results

Comparing human and machine visual perception can be challenging. In this work, we presented a checklist on how to perform such comparison studies in a meaningful and robust way. For one, isolating a single mechanism requires us to minimize or exclude the effect of other differences between biological and artificial and to align experimental conditions for both systems. We further have to differentiate between necessary and sufficient mechanisms and to circumscribe in which tasks they are actually deployed. Finally, an overarching challenge in comparison studies between humans and machines is our strong internal human interpretation bias. 

Using three case studies, we illustrated the application of the checklist. The first case study on closed contour detection showed that human bias can impede the objective interpretation of results and that investigating which mechanisms could or could not be at work may require several analytic tools. The second case study highlighted the difficulty of drawing robust conclusions about mechanisms from experiments. While previous studies suggested that feedback mechanisms might be important for visual reasoning tasks, our experiments showed that they are not necessarily required. The third case study clarified that aligning experimental conditions for both systems is essential. When adapting the experimental settings, we found that, unlike the differences reported in a previous study, DNNs and humans indeed show similar behavior on an object recognition task. 

Our checklist complements other recent proposals about how to compare visual inference strategies between humans and machines (Buckner, 2019; Chollet, 2019; Ma & Peters, 2020; Geirhos et al., 2020) and helps to create more nuanced and robust insights into both systems.


Testimonials


  • Such a nice optician. My grandson is only 4 and needs glasses. We were so sad, but he explained the issues, and we will follow up as he suggested.


    Anna P.

  • I had such a good experience with the clinic. Very friendly staff and doctor, did not have to wait for long to be called, and was treated respectfully. Thanks, American Family.


    June S.

  • Dr. Zurcher has gone above and beyond what any other eye doctor has ever done to figure out what is going on with my eyes. Very happy with American Family Vision.


    Christine R.

  • Family Vision Clinic changed our lives! My daughter was frequently car sick, and she was getting headaches every day, often painful enough that they brought her to tears. We saw a string of doctors and therapists, but we made no progress. Finally, we found Dr. Levi Zurcher and his eye therapist Rain. After ten weeks of eye therapy my daughter no longer gets headaches, and she no longer gets car sick. Daily tears are a thing of the past. I really can’t say enough about this clinic. It was fascinating to watch Dr. Zurcher work. For the first time, someone who knew what they were doing was intently studying my daughter, really trying to figure out all of her eye issues, and his therapist Rain is one of the most patient and lovely people that I’ve ever had the good fortune to meet. We’re finished with eye therapy—yay!—and I have switched to American Family Vision Clinic for all of my family’s other eye health needs.


    Lars Wulff

  • Very professional, yet kind and helpful. They do what they can to make the appointment comfortable. I was running a bit late, I made sure ti call. They were able to switch me with a patient who was already there, they treated me with respect, and helped my son have confidence by getting him the eye care he needed that day. Thank you so much! Would recommend to anyone. It's a blessing that they care enough to work with people who have all different types of insurance from work to state coverage.


    Justin E.

Blog

Atkinson Hyperlegible Font (1)

Atkinson Hyperlegible Font

Named after Braille Institute founder, J. Robert Atkinson, Atkinson Hyperlegible font is great for low vision readers.  In contrast to […]

Read More
displeased-grey-haired-man-suffers-headache-keeps-hands-head-reveal-pain-needs-painkillers-has-migraine-after-noisy-party-wears-formal-shirt-isolated-brown-wall

Binocular Vision Dysfunction (BVD)

Binocular vision refers to the ability of the eyes to work together. The term binocular vision dysfunction describes a misalignment […]

Read More
woman-putting-eye-drops-sore-eyes (1)

Contact Lens Discomfort Caused by Dry Eyes

Dry eye symptoms are the main reason that most people stop wearing contact lenses. There are many causes and reasons […]

Read More
see all blogs

Contact Us To Amplify Your EyeCare

Working Hours

Monday - Thursday
8:00 am - 5:15pm
friday-sunday
Closed
Location
400 Yauger Way SW. Bldg 1, Ste A Olympia, WA 98502
Website Accessibility Policy
Safety protocols page
phone-handsetarrow-uparrow-right linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram