Summary
The vOICe, notable for the spelling of “O I C”, is a program aimed to assist people with varying forms of blindness to see using an association of black and white images with soundscapes. The program was developed by Dr Peter Meijer, of the Netherlands, and is hosted at the SeeingWithSound.com website. As of now the vOICe can be run only on Windows and Android, but there has not been time to develop the programme for cross-platform devices.
Description
The vOICe takes advantage of a few bits of equipment for a person to be successful. A portable computer can be modified to function while the lid is closed. The user can run the application after securing their camera glasses mounted on their head to the computer by using a good-sized USB cable. The programme will start out by testing the headphones by speaking two words on each side. “Left, right.” By a complex set of arrays built into the programme, the application will take a video shot of what is in front of the camera from left to right, convert the result into a unique set of computer-generated sounds consisting mainly of tones and noise providing many textures. The higher set of frequencies correspond to activity in the upper portion of the camera and vice versa. The brighter an object is to the camera, the louder the sounds will appear. If the object gets closer to the camera, the background sounds will gradually diminish until the object is within full view. This is part of the ratio that a lot of people misconceive. It takes one second for a sound scape to be converted from the video, applying sixty-four pixels with a predetermined range of audible frequencies. Note, for a first-time blind person, they are not going to know what a pixel is. In brief, a pixel is a squared lined boarder that holds part of an image.
The vOICe can also be used to learn how to use the mouse. Using the mouse is like taking a probe across a big square grid, you learn to identify the sound of the mouse with the items on this grid. Then when you move the mouse, the sound moves in accordance to wave field holophony synthesis. When you find a major area of focus, you press the mouse to focus on the smaller area within that grid. You have more access to your focus on that area. When you are done, you focus on yet another smaller bit of detail within that grid. So it is like opening a huge box, then finding the box you want within that huge box. Then you open that box and then hunt for the other box, and then you open that box, and so on.
Along with the vOICe, researchers around the world have developed similar sensory substitution devices that serve the same or similar purpose as the vOICe. One example is the Hebrew University’s pentatonic EyeMusic image converter in C-major. Many researchers gather around certain places to discuss their beliefs about how people with any form of disability can benefit from technology adapted for such people. As of recent, there was a study that was published in the journal of Cognitive Science that showed who the vOICe could be of help to anyone who wants to see using their ears.
Purpose
With the ever-lasting work on regrowing nerves and other tissue in the eyes, it is hoped that sensory substitution devices can be used to encourage visual rehabilitation to train the brain to see either for the first time, or after a period of prolonged absence. There is talk about inventing transceivers that can transmit images from one brain to another brain by use of magnets. This way blind people have all the colours in their minds, and it will be up to them to make their own images in their heads after this new sensation is given to them.
Disadvantages and negativities
As of right now, the vOICe, or any form of sensory substitution, cannot be used for extreme applications due to the imposing of danger to the person using them. This includes, traveling in unfamiliar environments, crossing streets, etc. Each form of sensory substitution will have its own ups and downs, so it is important that the person finds the one that best fits their needs. Currently, all devices are set to be easily customised for the end-user, though that may get overwhelming since there are many options to change. The idea is to simulate a form of natural eyesight, which cannot be customised, save for putting on special glasses and such.
Comments
I have been using the vOICe off and on, but I find it quite confusing to grasp all at once. Throughout experimentation and research, I found out that the vOICe is primarily focusing on the association of visual images to audible soundscapes, not so much with colours to sounds, like some of the other devices do. This is because those devices may not give you the shape of an object, just the colour of what it is. Another problem is that there is no distinction between knowing the different variations of grey with the use of pitch, since that is already being used for knowing the height of an object is. I found out that most people are familiar with associating colours with musical keys, but those musical keys are what give us specific emotions that can be correlated to colours. Another possibility is the advent of certain cadences that resolve to a certain key, or when combining three or more colours, they will create a new harmony that can give us an emotion, such as pastel colours.
Suggestions and improvements
Most of these have already been considered and are being worked on, however they will still be addressed. As of now the most effective way to receive sound input is through stereo headphones. A possible suggestion is to use a more complex system that will use wave-field synthesis technology to enhance the spacial awareness of the person. Instead of having the sounds tell you how high an object is, the sounds will come from the upper portion of the set, and vice versa. This way, the pitches of sound can be used for other applications, such as identifying colours, shades, tints, etc.
A discussion regarding stereoscopic vision brought the possibility of wearing two cameras, one above each eye, that would move in accordance to eye movements of the person (if they do not suffer from nystagmus) or some other neurological disorder. This can be accompanied with the surround-sound system, so that the left side is depended on the left camera while the right side is depended on the right camera.
The ability to focus and keep track of an object with the camera would be helpful to know when a particular object is moving. This would mean stopping the scanning process and have the programme generate a sound scape for that image in real time. When the image shifts to one side, the sounds will move in the direction that it went. This sort of thing can be found in many forms of audio-gaming. Focusing on the object itself is to hear the sound scape of the object while moving the camera to study the details of each side of the object could be helpful. For example, you want to study a painting. Using the vOICe, you can stop the scanning and concentrate on the paint. Using a probe-like fashion, you can manually scan the painting with the camera(s) to hear each sound scape change in volume, pitch, timbre and texture.
Brain wave entrainment can be added to synchronise the frequency bands of a person’s brain, that way they can be more aware of the area around them. The concept “qualia” refers to a state of meditative consciousness, such as twilight or dawn. Such things can be achieved through not only visual brain wave synchronization, but auditory as well. This includes, beatings (binaural or monaural), or isochronic pulses. It also consists of certain chords not found in forms of conventional music, and therefore has no organized time and sound.
One possibility is to use the colour functions as part of brain wave entrainment. The soundscapes can indicate the black-and-white area surrounding the person, while the brain wave soundscapes fill them in as to what the colours are around them.
Current research has not dove much into the latter area, but it is said that it helps in making a person feel relaxed and aware. It is also said that people with music training have higher performance with auditory visual stimulations.
To widen the number of user’s interest in the vOICe, it is recommended that more platforms be included.
Leave a Reply