Fiddler Crab Eye View Inspires Gwangju Institute of Science and Technology Researchers to Develop Novel Artificial Vision
The researchers develop an amphibious artificial vision system with a panoramic field-of-view based on the Fiddler crab’s eye structure.
Artificial vision systems are implemented in motion sensing, object detection, and self-driving vehicles. However, they are not suitable for changing external environments and are limited to a hemispherical field-of-view (FOV). Addressing this issue, researchers from GIST have now developed a novel artificial vision with 360° FOV that can image both terrestrial and aquatic environments. The system, modeled after the eye structure of the fiddler crab, could help realize the all-weather vision and panoramic object detection.
Artificial vision systems find a wide range of applications, including self-driving cars, object detection, crop monitoring, and smart cameras. Such vision is often inspired by the vision of biological organisms. For instance, human and insect vision have inspired terrestrial artificial vision, while fish eyes have led to aquatic artificial vision. While the progress is remarkable, current artificial visions suffer from some limitations: they are not suitable for imaging both land and underwater environments, and are limited to a hemispherical (180°) field-of-view (FOV).
To overcome these issues, a group of researchers from Korea and USA, including Professor Young Min Song from Gwangju Institute of Science and Technology in Korea, have now designed a novel artificial vision system with an omnidirectional imaging ability, which can work in both aquatic and terrestrial environments. Their study was made available online on 12 July 2022 and published in Nature Electronics on 11 July 2022.
“Research in bio-inspired vision often results in a novel development that did not exist before. This, in turn, enables a deeper understanding of nature and ensure that the developed imaging device is both structurally and functionally effective,” says Prof. Song, explaining his motivation behind the study.
The inspiration for the system came from the fiddler crab (Uca arcuata), a semiterrestrial crab species with amphibious imaging ability and a 360° FOV. These remarkable features result from the ellipsoidal eye stalk of the fiddler crab’s compound eyes, enabling panoramic imaging, and flat corneas with a graded refractive index profile, allowing for amphibious imaging.
Accordingly, the researchers developed a vision system consisting of an array of flat micro-lenses with a graded refractive index profile that was integrated into a flexible comb-shaped silicon photodiode array and then mounted onto a spherical structure. The graded refractive index and the flat surface of the micro-lens were optimized to offset the defocusing effects due to changes in the external environment. Put simply, light rays traveling in different mediums (corresponding to different refractive indices) were made to focus at the same spot.
To test the capabilities of their system, the team performed optical simulations and imaging demonstrations in air and water. Amphibious imaging was performed by immersing the device halfway in water. To their delight, the images produced by the system were clear and free of distortions. The team further showed that the system had a panoramic visual field, 300o horizontally and 160o vertically, in both air and water. Additionally, the spherical mount was only 2 cm in diameter, making the system compact and portable.
“Our vision system could pave the way for 360° omnidirectional cameras with applications in virtual or augmented reality or an all-weather vision for autonomous vehicles,” speculates Prof. Song excitedly.
And it might be soon!
Reference
Authors: Mincheol Lee (1,2), Gil Ju Lee (3,4), Hyuk Jae Jang (3), Eehyung Joh (1,2), Hyojin Cho (1,2), Min Seok Kim (3), Hyun Myung Kim (3), Kyeong Muk Kang (3), Joong Hoon Lee (3), Myungbin Kim (1,2), Hongwoo Jang (5), Ji-Eun Yeo (3), Frédo Durand (6), Nanshu Lu (5,7,9,10,11), Dae-Hyeong Kim (1,2,8), and Young Min Song (3)
Title of original paper: An amphibious artificial vision system with a panoramic visual field
Journal: Nature Electronics
DOI: https://doi.org/10.1038/s41928-022-00789-9
Affiliations:
- Center for Nanoparticle Research, Institute for Basic Science (IBS)
- School of Chemical and Biological Engineering, Institute of Chemical Processes, Seoul National University
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology
- Department of Electronics Engineering, Pusan National University
- Texas Materials Institute, University of Texas at Austin
- Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology
- Department of Aerospace Engineering and Engineering Mechanics, The University of Texas at Austin
- Department of Materials Science and Engineering, Seoul National University
- Department of Electrical and Computer Engineering, The University of Texas at Austin
- Department of Biomedical Engineering, The University of Texas at Austin, Austin
- Department of Mechanical Engineering, The University of Texas at Austin
About the Gwangju Institute of Science and Technology (GIST)
The Gwangju Institute of Science and Technology (GIST) is a research-oriented university situated in Gwangju, South Korea. Founded in 1993, GIST has become one of the most prestigious schools in South Korea. The university aims to create a strong research environment to spur advancements in science and technology and to promote collaboration between international and domestic research programs. With its motto of “A Proud Creator of Future Science and Technology,” GIST has consistently received one of the highest university rankings in Korea.
Website: http://www.gist.ac.kr/
About the authors
Young Min Song is currently a Professor in the School of Electrical Engineering and Computer Science at Gwangju Institute of Science and Technology (GIST), Korea. He received his Ph.D. in Information and Communications from GIST in 2011. From 2011 to 2013, he was a postdoctoral research associate in the Department of Materials Science and Engineering at the University of Illinois at Urbana-Champaign (UIUC), USA. Over the decades, he has predominantly focused his efforts on developing bio-inspired optics and photonics. His group is developing advanced optoelectronic sensors and systems, multifunctional nanophotonics, and optical healthcare systems.
Dae-Hyeong Kim obtained Ph. D. degree in Materials Science and Engineering from University of Illinois at Urbana Champaign (UIUC) in 2009. He was a post-doctoral research associate at UIUC from 2009 to 2011. He joined Seoul National University in 2011 and is currently a Professor in the School of Chemical and Biological Engineering at Seoul National University. Since 2017, he has also served as an associate director of Center for Nanoparticle Research at Institute for Basic Science (IBS). He has been focusing on the research of nanomaterials and deformable devices and their application to bio-integrated and bio-inspired electronics.
Mincheol Lee is a post-doctoral research associate at Seoul National University. He received his Ph. D. degree in the School of Chemical & Biological Engineering from Seoul National University in 2021. He has been focusing on the research of stretchable electronics based on ultrathin semiconductor films and their application to wearable, implantable, and bio-inspired electronics.
Gil Ju Lee is currently serving as an Assistant Professor in the Department of Electronics Engineering at Pusan National University since 2021. He has intensely focused on the research of advanced photonics and optics for next-generation optoelectronics and optical systems. Before coming to Pusan National University, he completed his Postdoctoral training and doctoral course at Gwangju Institute of Science and Technology (GIST).
Hyuk Jae Jang received his B.S. from Department of Physical and Semiconductor Science at Dongguk University and is a Ph. D. candidate under the guidance of Prof. Young Min Song in GIST. His current research interests focus on bio-inspired nano-optics and imaging device systems.
