The pro gamer who has to rely upon sound alone

In the competitive world of professional gaming, one gifted player goes by the username Rattlehead.

At the tournaments he attends in the US, his opponents quickly spot that he, real name Carlos Vasquez, is, by his own description, “completely blind”.

They then let their guard down, wrongly thinking that they are set for an easy game of popular fighting series Mortal Kombat. And he often beats them.

Instead of being able to see the game, Carlos instead relies upon hearing it to make his attacking and defensive moves. “I’m able to pick up on the specific sounds of the characters,” he says.

From Houston, Texas, Carlos started playing video games at the age of six in 1992. But when he was 11 he was diagnosed with a type of glaucoma, an eye condition that slowly took away his sight.

“It wasn’t an overnight situation,” says Carlos, now 37. “[For the first years] I was still able to see rough outlines of objects, but not fully detailed anymore.

“But over time my vision became worse and worse, to the point where I was around 24-years-old and completely blind.”

At the time he found that most mainstream games were almost impossible to play. Features that help people with a sight impairment to play, known as gaming accessibility, were not around.

Such features include screen readers, where the game verbally describes what is happening.

Back in his 20s, Carlos was able to play some fighting games, and in particular Mortal Kombat.

“It wasn’t because Mortal Kombat had accessibility features [at the time],” he says. “It was just because players like myself were able to pick up on the [distinct] sounds. It allowed us to play the game as much as possible.”

For example, he recognised that the character who started on the right side of the screen would grunt at a slightly higher pitch than the one on the left. This allowed him to work out which character he was playing.

Fast forward to 2023, and a growing number of games have accessibility features, including Forza Motorsport, Diablo 4, The Last of Us, Hearthstone, Street Fighter 6, and the latest edition of Mortal Kombat.

Forza Motorsport has a function called Blind Driving Assists whereby the visually impaired are verbally told of forthcoming bends on the track and their sharpness. Various noises or audio cues indicate things such as a car’s speed, whether it is facing the wrong way, or when to change gear.

For Mortal Kombat a whole host of additional audio feedback is now available, such as letting you know the distance between fighters, or if an opponent ducks.

Forza Motorsport also includes the latest development in games accessibility – the increased use of a technology called spatial audio.

This uses specifically directed sound to significantly help visually impaired players work out their precise location on the screen.

Prof Brian Smith, an expert in the field, explains how it works: “When you hear a sound [in real life], such as person speaking from a certain location around you, the physical waveform of the sound reaches your two different ears at slightly different times, at slightly different volumes.

“Then the sound will go through your head to reach the other ear, and so that signal is also slightly transformed. Your brain is very good at getting those signals at different timings and saying: ‘Oh, that means the sound is coming from this location, and it’s a single sound’.”

Prof Smith’s work at the Department of Computer Science at the University of Columbia in New York is focused on taking that real world, directional principal of hearing, and applying it to the audio produced by a computer game.

The technology works via stereo speakers and headphones, but users get better results from specialist spatial audio equipment.

Prof Smith adds that it can be so effective in helping people with impaired vision control characters in a computer game that the difficulty is finding a balance between making the games accessible without taking away the challenge.

Forza Motorsport is one of the games that he and his team helped work on.

While the major games studios are now embracing accessibility, it was previously independent hobbyist developers who often led the way, including some that are visually impaired themselves.

Seven years ago, The AbleGamers Charity, which campaigns for more computer games that can be played by people with disabilities, asked game developer Rockstar how much it would cost to make its Grand Theft Auto V title accessible.

The firm is reported to have replied that it would cost a whopping $128m (£100m), which it considered too expensive.

As a result of this, an independent software developer called Liam Erven decided to rise to the challenge, and develop a software ad-on, also known as a mod, to make Grand Theft Auto more playable for people with impaired vision.

Liam, who has been blind since birth, set to work with a team of collaborators to create their mod, which is called Grand Theft Accessibility.

Features they were able to add include sound clues to give players information about nearby objects, vehicles and pedestrians. Users also get audio feedback about their location and direction, and what weapon they have selected.

It was all done for a fraction of the cost estimated by Rockstar.

Liam says that while the ad-on “isn’t anywhere near perfect” it is still “really fun to play”, and has garnered a solid user base.

Prof Smith says that such independent developers have played a big part in raising awareness, and “waking up” the big gaming studios so that they realise improving accessibility for blind players “may not be as hard as it seems”.

He adds, however, that the gaming industry still doesn’t have many blind people represented. “And so folks in the industry don’t realise, or I think they have a misinformed idea, that blind game accessibility is an oxymoron.”

New Tech Economy is a series exploring how technological innovation is set to shape the new emerging economic landscape.

However, things are continuing to improve. Carlos Vasquez is today not just beating opponents on Mortal Kombat.

He now also works for the game’s developer, NetherRealm Studios, as an accessibility consultant. He says he’d like gaming firms to employ more people like him.

“In the past, developers basically guessed what people with disabilities were looking for, rather than reaching out to people and consulting with them,” he says. “Companies need to reach out to people who are literally living this every day and get their feedback.

“Whether it’s in the studio, or getting a chance to have early access for testing, something we’re starting to see more of. But I think what the industry overall needs to do better is being more transparent, about whether their game is going to have accessibility features or not.”