This document aims to serve as a guide to inform markets on Track AI technology specifications. This document is for internal use only but can be used to answer questions from media.
Background
Huawei has partnered with leading research institute, IIS Aragon and DIVE Medical to co-create an AI-powered assessment for early signs of visual impairment.
For the last few years, DIVE Medical has worked on the development of a new medical device to provide an automated, fast, and accurate eye test, even in patients unable to communicate their needs.
DIVE (Devices for an Integral Visual Examination) is specially designed to explore vision in children, including infants from six months of age and children with development issues. Earlier diagnosis of any visual problem allows earlier treatments, decreases its consequences and helps to orientate rehabilitative and educational strategies.
DIVE Medical has developed tests to identify visual problems in children by exploring gaze patterns on scientifically designed visual stimuli. Accurate and objective data of the gaze position of the patient is obtained by means of an eye-tracker.
Further information on how this works is included throughout this document.
How it Works – The Device
- Eye monitoring
The HUAWEI Matebook E Hybrid laptop is connected to the DIVE device and a HUAWEI smartphone (HUAWEI P30 Pro). Once the connection is made, the DIVE software is run. This software consists of a suite of tests that adapt to the capabilities of each patient for fast and accurate evaluation of their visual functions.
From this data, the software creates a gaze pattern which is then passed onto the smartphone app via Bluetooth. This can then be shared with the patient’s doctor who can use the data to determine what specialist care is needed.
Normal gaze pattern Abnormal gaze pattern
- Smartphone app
The gaze pattern of each eye played on the Matebook E Hybrid is assessed by the smartphone (Huawei P30) running the DIVE app. It is here that the AI processing via the app integrates an algorithm, which has been created through the machine learning of a large dataset of multiple patient gaze patterns.
The Smartphone app has been created using TensorFlow and HUAWEI’s own HiAi Developer kit. This allows the information to be processed on the smartphone app using the NPU. At the heart of the app is an AI based algorithm which has been taught using a vast data set of gaze patterns to understand what a healthy gaze pattern is, and therefore when there is a visual impairment highlighted by an abnormal gaze pattern.
This is what the app on the smartphone will look like:
How it Works – The Software
The processing power required to carry out this exercise is extremely intense and therefore is being rolled out the Dual NPU of the Kirin 980.
What this means is HUAWEI does not have to send patient data off the device itself to support Cloud-AI processing. This cuts the time it takes considerably and gives doctors who are based in remote villages with little or no connection to the internet, more time to see more patients, more power and more freedom to continue helping their patients and improving their overall health and wellbeing.
- TensorFlow
Track AI builds on the capabilities of Google’s TensorFlow, an open-source platform, and Huawei HiAi to create a machine learning system on a smartphone using its breakthrough Kirin 980 chip, an AI processing powerhouse that is defining the future of smartphone AI.
Two dedicated Neural Processing Units (Dual NPU) built inside the Kirin 980 support on-device AI which means the AI functions are processed on the smartphone itself, significantly speeding things up while also preserving user privacy. This Dual Neural Processing Unit also helps to make the Smartphone's AI features smarter over time as well as increase daily performance and efficiency.
- HiAI developer kit
HiAI is one of the world's first mobile AI computing platforms with a dedicated Neural-network Processing Unit (NPU).
The HiAI API library, released as a unified binary file, is set to enable fast neural network-based computing through HiAI Heterogeneous Computing Platform, which is integrated within Kirin SoC.
With the HiAI APIs, developers can focus on developing new AI applications instead of focusing on performance tuning.
Track AI uses a machine learning algorithm and the HiAI platform to create a set of HUAWEI smartphone-based assessments for visually impaired children. The sophisticated HiAI platform performs at an ultra-high function resulting in a greater degree of accuracy.
Types of visual impairments the technology helps to identify
The primary purpose of Track AI is to help identify visual impairments in children, even as young as six months old. The app is set up to assess for the following:
- Visual function including visual acuity (clarity of vision)
- Oculomotor control (the assessment of how the child fixates and moves the gaze on a picture
- Contrast sensitivity or colour perception
Track AI can identify any disorder affecting visual function, even in extremely subtle cases, which would remain undiagnosed with traditional assessments.
How this helps doctors
The Track AI system is designed to examine several aspects of visual performance, which cannot be assessed by traditional examinations, especially when the patients is too young.
The Track AI application helps doctors facilitate highly skilled diagnosis through the contained AI algorithm inside the app. This means that a local doctor can get a good idea of which patients need expert care using this tool and can easily identify any serious impairments accurately as well as predict future issues.
FAQs
Why have Huawei and DIVE partnered?
Huawei has partnered with leading research institute, IIS Aragon and DIVE Medical to co-create an AI-powered assessment for early signs of visual impairment. The project, Track AI is part of Huawei’s wider commitment to push the boundaries of what is humanly possible and use AI for good. This project makes it easier to detect visual impairments in a faster and more efficient way through AI technology.
What problem is Huawei and DIVE trying to solve?
The World Health Organization estimates in 19 million the children in the world with a visual impairment. Reports of the global magnitude and causes of visual impairments confirm that 70-80% of all visual impairments are either preventable or curable. Most of these children will remain undiagnosed for years, leading to consequences on their vision, general development, educational opportunities, social life and prospects.
Most of these children will remain undiagnosed for years, leading to consequences on their vision, general development, educational opportunities, social life and prospects.
The DIVE device, powered by Huawei’s AI technology, is a preventative measure to help solve a worldwide problem.
What is Track AI?
The primary purpose of Track AI is to help identify visual impairments in children, even as young as six months old. The app is set up to assess for the following:
- Visual function including visual acuity (clarity of vision)
- Oculomotor control (the assessment of how the child fixates and moves the gaze on a picture
- Contrast sensitivity or colour perception
Track AI can identify any disorder affecting visual function, even in extremely subtle cases, which would remain undiagnosed with traditional assessments.
Track AI uses a machine learning algorithm and the HiAI platform to create a set of HUAWEI smartphone-based assessments for visually impaired children. The sophisticated HiAI platform performs at an ultra-high function resulting in a greater degree of accuracy.
What is the motivation behind Track AI?
During the last few years, DIVE’s researchers have worked on the development of a new medical device to provide an automated, fast, and accurate exploration of the visual function in patients as young as six month and those who can’t communicate. DIVE is specially designed to explore vision in children, including infants from six months of age.
Earlier diagnosis of any visual problem allows earlier treatments, decreases its consequences and helps to orientate rehabilitative and educational strategies.
What are the causes of visual impairment in childhood? What visual impairments can it help to detect?
The major causes of visual impairment in childhood vary for each child and can even be dependent on factors such as region of the world. For example, in developing countries nutritional and infectious conditions are the main causes for blindness. Vitamin A deficiency, corneal scarring related to measles or neonatal conjunctivitis are common in certain cases seen in Asia and / or Sub Saharan Africa. Whilst Retinopathy of prematurity is causing major visual impairment in middle income countries. Nevertheless, the aetiologies of severe visual deficiencies in developed countries are mainly due to hereditary and perinatal factors.
Uncorrected refractive errors are causes of less severe visual impairment, but it is currently affecting around 12.8 million children aged from 5 to 15 years in the world, with the highest prevalence reported in urban areas of China and south-east Asia. Increasing rates of myopia has become a concerning public health problem in other regions. Finally, amblyopia is the result of different conditions that cause visual impairment, but avoidable if treated in time. Therefore, screening for amblyopia is one of the most important goals in paediatric ophthalmology programs.
How does the DIVE device work?
DIVE has developed tests to identify visual problems in children by exploring gaze patterns on scientifically designed visual stimuli. A huge amount of accurate and objective data of the gaze position of the patient is obtained by using DIVE’s eye tracking capabilities. This data even includes the patient’s visual function, related to visual acuity, contrast sensitivity and colour.
DIVE built on a HUAWEI Matebook E Hybrid laptop is connected to an eye tracking monitor and a HUAWEI P30 Pro. Once the connection is made, the DIVE software is run. This software consists on a suit of tests that adapt to the capabilities of each patient for fast and accurate evaluation of their visual functions. From this data, the software creates a gaze pattern which is then passed onto the smartphone app via Bluetooth. This can then be shared with the patient’s doctor who can use the data to determine what specialist care is needed.
How is the data then monitored and evaluated?
The gaze pattern of each eye played on the Matebook E Hybrid is assessed by the smartphone running the DIVE app. It is here that the AI processing via the app integrates an algorithm, which has been created through the machine learning of a large dataset of multiple patient gaze patterns.
The Smartphone app has been created using TensorFlow and HUAWEI’s own HiAi Developer kit. This allows the information to be processed on the smartphone app using the NPU. At the heart of the app is an AI based algorithm which has been taught using a vast data set of gaze patterns to understand what a healthy gaze pattern is, and therefore when there is a visual impairment highlighted by an abnormal gaze pattern.
How is Huawei’s HiAI used to power DIVE?
Analysing gaze data is a challenging problem. Starting from the DIVE platform and its gaze analysis software, the start-up is now incorporating artificial intelligence, thanks to Huawei’s HiAI platform. This combination unleashes the potential to not only measure visual function, but to also estimate the probability of the patient having issues with their vision in the future
Developing a system based on AI requires gathering data from thousands of children. During this unique collaboration between Huawei and DIVE, 18 ophthalmologists from 5 different countries across 3 continents will collect data from children either with normal visual development or with a wide range of visual impairment.
With this information, and the use of Huawei’s artificial intelligence technology, DIVE aims to find gaze patterns in the way children look at visual stimuli and relate them to visual development and to certain specific impairment. This will enable fast and reliable detection of visual impairments
The unprecedented capabilities of Huawei’s P30 smartphone allow DIVE technology to reach out and be useful across the world, from an ophthalmologist in a hospital in the UK to a family doctor in a remote area of Mexico.
What is the vision for doctors and ophthalmologists?
The Track AI system is designed to examine several aspects of visual performance, which cannot be assessed by traditional examinations, especially when the patients is too young.
The Track AI application helps doctors facilitate highly skilled diagnosis through the contained AI algorithm inside the app. This means that a local doctor can get a good idea of which patients need expert care using this tool and can easily identify any serious impairments accurately as well as predict future issues.
Has this been done before?
This project is unique since it aims to identify visual impairments in infants on a world scale and multi-ethnic approach, catering to every type of visual impairment.
Which countries are currently collecting data for the project?
Developing a system based on AI requires gathering data from thousands of children, and data is currently being collecting by the research centres from testing in five countries across 3 continents (China, Mexico, UAE, Spain and UK).
