New technology tracks eye gaze and body pose to help pupils engage in learning
New technology which tracks the eye gaze and body pose of children to understand and improve their engagement with learning has been designed in collaboration with Å·ÃÀ¾ÞÈé (NTU).
By Hayley Chipman | Published on 18 June 2019
Categories: Press office; School of Science and Technology;

The innovative digital system, calledMaTHiSiS, understands the emotional state of the student, such as whether they are engaged, frustrated or bored. It also assesses the student’s skill level and presents them with personalised electronic materials to continue their learning.
Part of a European-wide project, the UK study has focused on students with learning disabilities and autism. NTU was the lead partner in understanding the requirements of these students so that their needs could be factored into the design of the system, such as any visual, physical or cognitive impairments they might experience, and their learning goals.
The University also had a significant input into computer methods designed to understand the emotional state of learners. These include tracking their interaction with the system, such as their mouse movements, or patterns and speed of typing, and relating this to whether they are engaged or not.
MaTHiSiS challenges preconceived ideas of the abilities of students with a learning disability and can reveal skills, knowledge and abilities that teachers had not otherwise understood or realised.
The UK partners, including NTU, University of Å·ÃÀ¾ÞÈé and University of East London, worked together to evaluate the results of pilots which took place in schools across Italy, Spain and the UK.
David Brown, Professor of Interactive Systems for Social Inclusion at Å·ÃÀ¾ÞÈé, said: “The MaTHiSiS system moves beyond current practice in using AI in Education where learner-facing tools use formative assessments and diagnostic tests to personalise the learning experience. We also seek to understand the emotional state of learners as this will have a great impact on the quality of their learning experience.”
The development of the MaTHiSiS system relied heavily on the input of teachers, teaching assistants and students from several European countries so that the end product was something the teachers and students could benefit from.
Funded by the Horizon 2020 Framework Programme of the European Union and developed in collaboration with 18 European partners, the study involved 34 teachers and 67 children between the ages of 6 – 16 who all took part in analysing and testing the efficiency of the system.
Findings from the pilot showed that teachers frame MaTHiSiS as a change in their teaching practice and compared it with the introduction of other ground-breaking technologies. They felt that MaTHiSiS will free them up to be able to give more support and encouragement to the learner and target their support to those learners who needed it most in a busy classroom where one to one support is at a premium.
Gosia Kwiatkowska, acting director of the RIX Centre, University of East London, said: “We are getting more diverse students in our classrooms and that requires diverse teaching. Our project addressed the real issue of teachers not having enough capacity to attend to each child’s individual learning needs and as a result some of the students are left out and are not supported to develop their full potential.
“A system that can detect a learner’s state of flow can provide a wonderful addition to any classroom. I can see that in the future there will be lots of personalised technologies available to teachers and students which will make learning much more effective, engaging and fun. The addition of tailored learning materials mean that students can continue their schoolwork at home.”
Professor of Health Psychology and Learning Disabilities at the University of Å·ÃÀ¾ÞÈé, Penny Standen, said: “MaTHiSiS is successful at monitoring the emotional state of the learner, especially their level of engagement. The significance of this is that engagement is considered to be the single best predictor of learning for children with intellectual disabilities.
“The sensitivity with which it monitored the learner’s progress, made it particularly suited to learners with special needs where achievements are made in small steps so aren’t always picked up by the monitoring derived from national curriculum levels.”
MaTHiSiS is for tutors, parents or caregivers, learners and teachers.
The MaTHiSiS partners are currently investigating ways of rolling out the system for use in education and industrial training. They are also proposing ways in which the system can be used as a research tool in other projects, and extending the methods developed in this project for use in other AI Tools in Education to predict the engagement of pupils with a learning disability to provide personalised learning pathways.
Notes for Editors
Press enquiries please contact Dave Rogers, Head of Communications, on telephone +44 (0)115 848 8782, or via email; or Chris Birkle, Press Officer, on telephone +44 (0)115 848 2310, or via email.
Å·ÃÀ¾ÞÈé (NTU) was named University of the Year 2017 at the Times Higher Education Awards, and Modern University of the Year in the Times and Sunday Times Good University Guide 2018. These awards recognise NTU for its high levels of student satisfaction, its quality of teaching, its engagement with employers, and its overall student experience.
NTU has been rated Gold in the Government’s Teaching Excellence Framework – the highest ranking available.
NTU is one of the largest UK universities. With 30,000 students and more than 4,000 staff located across four campuses, the University contributes £900m to the UK economy every year. It is one of the UK’s most environmentally friendly universities, containing some of the sector’s most inspiring and efficient award-winning buildings. 96% of its graduates go on to employment or further education within six months of leaving.
Our student satisfaction is high: NTU achieved an 88% satisfaction score in the 2018 National Student Satisfaction Survey.
The University is passionate about and its extensive outreach programme is designed to enable Å·ÃÀ¾ÞÈé Trent to be a vehicle for social mobility. NTU is among the UK’s top five recruiters of students from disadvantaged backgrounds.
NTU is home to world-class research, and won The Queen’s Anniversary Prize in 2015 – the highest national honour for a UK university. It recognised the University’s pioneering projects to improve weapons and explosives detection in luggage; enable safer production of powdered infant formula; and combat food fraud.
With an international student population of over 3,000 from around 100 countries, the University prides itself on its .