Researchers and designers are bringing poetry to life for the visually impaired in Cali, Colombia

A book of recordings of poetry set to music, suitable for the blind and others with visual disabilities, is the latest project of a research group led by the Faculty of Engineering of the Universidad del Valle (Univalle).

Professor Paola Johanna Rodríguez Carrillo, leader of the Camaleón research group at the Computer and Systems Engineering School (EISC), explained that the book represents another big step forward in the group's journey in human-computer interaction, especially with communities with disabilities.

“Poetry audio collections are few and far between compared to audiobooks, and there is also something lost when poetry is read by an artificial voice,” Professor Rodríguez said.

  

Photo: A prototype of the book of poetry for the blind. Credit: Professor Javier Reyes

A Book Of Poetry For The Blind

Professor Javier Mauricio Reyes, a designer with a doctorate in computational science and a researcher from the Camaleón group, said that in his doctorate he did work with tangible user interfaces (TUI): objects or pieces you can touch and/or pick up, which also have computational capacity.

"We made some prototypes where we involved a visually impaired person who was with us in this process, even in the testing of the prototypes... This is where our interest in this area was born," Professor Reyes said.

According to the  2018 Colombian census, there are almost two million people in Colombia with some visual disability, more than 4% of the population.

The professor said that at the moment the group is developing a multi-use book, with braille and illustrations, accompanied by a mobile application with words read aloud and music, as well as a video. Microchips inside the book interact with the app to coordinate each page.

"The theme of working with poetry is... to interest the visually impaired in the artistic expression of poetry," the professor said, adding that the idea was to combine the two ways of experiencing poetry: reading Braille at the same time listening to words and music.

Professor Rodríguez explained that the project focused on fifth and sixth grade students, between 10 and 14 years old — particularly those with visual disabilities.

 "They know how to use a cell phone, they already know how to read braille," Professor Rodríguez said.

“We chose to work with poetry because it is not often that you find it in audio form,” she said, adding that they are working with a music student for the musical composition.

"Transmitting information with music can be very powerful, they can understand the emotion behind the words of poetry," Professor Rodríguez said.

Photo: Examples of interactive maps made by engineering students. Credit: Andrew James/NCC/Univalle.

 

Step By Step

The Camaleón research group was formed in 2012 and specializes in the interaction between people and the virtual world.

In her office at the School of Systems and Computing, Professor Rodríguez showed off different student projects by placing them, one by one, across her desk: Firstly, brightly colored maps of Colombia, and then a set of trading cards.

“The idea behind the colors of the maps is to produce something that can be of interest to all boys and girls,” explained Professor Rodríguez, adding that one objective of these projects is for children with disabilities to feel more integrated into their society.

With the maps, different points of historical interest were represented by a series of raised bumps: touching each triggers a different action.

"This can activate a video or audio in a mobile application," explained the Professor.

With the trading card game, each one has an image of a historical figure from the indigenous Calima culture and also Braille printed on them. The cards also have an RFID chip that can be read by a mobile device.

“We have access to a Braille printer from a university library,” the professor said.

The researchers say that for now they are focusing on visual impairment, but with the interest of working with hearing disabilities, the mobility disabilities and then growing  gradually from there.

READ MORE from the School of Systems and Computional Engineering: Women in STEM are the stars of Engineering Week 2021 at Universidad del Valle

A trading card with an RFID chip and Braille printing. Credit: Andrew James/NCC/Univalle

Mixing Art and Science

Professor Rodríguez explained that the project involves both engineers and designers to produce an educational product.

“Our group has both design faculty from the Faculty of Integrated Arts and engineers from the School of Systems and Computing,” said Professor Rodríguez.

Professor Reyes added that Camaleón projects are an opportunity to open an exchange of knowledge between engineers on one hand and artists and designers on the other.

"Engineering students can understand what the designer does and in the same way the designer can learn what the engineer does," Professor Reyes said. 

Professor Liliana Esther Machuca Villegas, who is also in the Camaleón group and did her doctorate in engineering, said that in the management of this type of project it is important to consider social and human factors to have better cohesion and performance of the work team.

"We are already studying this effect," said Professor Machuca.

If you would like to contact the researchers or learn more about the project, write to us at the Communications Office of the Faculty of Engineering: comunicaingenieria(at)correounivalle.edu.co.

Banner photo: Professor Paola Johanna Rodríguez Carrillo, director of the Chameleon research group of the School of Systems and Computing Engineering (EISC). Credit: Andrew James/NCC/Univalle.