AI knows what you type by simply listening

Researchers have trained an AI deep learning model to detect computer keyboard keystrokes with up to 95% accuracy. While there is room for improvement, the core mechanic has an AI listen to the keystrokes of a computer keyboard using conventional microphones, for instance of a modern smartphone, to record keystrokes.

The researchers Joshua Harison, Ehsan Toreini and Maryam Mehrnezhad published the research paper A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards this week.

To demonstrate the theory, the researchers trained an AI model on a MacBook Pro; this was done by pressing keyboard keys multiple times so that the AI model could learn the different sounds of the keypresses. They analyzed the waveforms of each of the keys pressed and noticed that each keystroke had a unique waveform when recorded.

Once trained, the AI managed to identify keystrokes with a 95% accuracy on the test MacBook Pro device using a conventional smartphone microphone. The researchers noted that accuracy would fall to 93% if the training was done via Zoom and that it would decrease even further when using VOIP.

The model supports local and remote spying provided that it has an audio feed that picks up the typing of the target and that it has been trained properly. One of the main downsides of the potential attack is that it needs to be trained on individual keyboards before it can detect keystrokes with accuracy. Also, the attack works best when mechanical keyboards and other noisier keyboards are used and less well when quieter keyboards are used by a target.

The researchers note that threat actors could use AI in targeted attacks to obtain passwords and other sensitive information, for instance during video calls.

Computer users may reduce the likelihood of successful attacks by changing their typing style. While that may be difficult, switching to touch-based keyboard input at random did reduce the accuracy of detection according to the researchers.

Other mitigation techniques include relying on random passwords instead of full word passwords, using autotype for passwords and other sensitive data instead of typing manually, or playing "defensive" sounds during video conferencing calls.

One of the best methods to thwart attacks was to add fake keystrokes, but this mitigation technique may not be applicable in all scenarios as it may impact the usability on the user's end.

Closing Words

Attacks using the described AI model to identify keystrokes using audio feeds should not unsettle users at this time. While there is a probability of target attacks against high value targets, large scale attacks do not appear feasible at this time.

Now You: what type of keyboard do you use?

Advertisement