Sound Design and the Future of Healthcare IT

Craig Joseph, MD
By: Craig Joseph, MD
Date: June 20, 2019

I recently read an article in Wired magazine about what sound design is with respect to information technology (IT) and why sound design is so important. I admit that I’ve rarely thought about this field of software development, but it really is important. What exactly is sound design? The author introduces the concept this way: “How do you know whether Siri heard you? Just as in human interaction, good communication is about the flow of conversation, the ongoing exchange of information.” Sound designers try to ensure that the two-way communication between humans and technology works, intuitively and naturally.

Sound Design in HealthcareMany physicians dream about the day that they’ll be able to largely disconnect from the keyboard and mouse and have documentation and ordering occur magically. Think about a doctor’s office in George Jetson’s day. As the doctor is speaking with the patient, a hidden microphone records the back and forth, identifying who is talking and what they’re saying. A nearby large video monitor shows a concise, clinically-focused progress note. The doctor narrates the physical exam as it’s performed, essentially telling the patient what the findings are, but at the same time, giving the magical computer the necessary data to compose a note.

Discrete data are put in the appropriate parts of the electronic health record (EHR) and a human-readable note is written. The doctor then says, “Based on your fever, cough, and the crackles I hear in your lungs, I think you’ve got community-acquired pneumonia. Let’s start you on the regular adult dose of amoxicillin, and I want you to go home with my standard pneumonia instructions. Let me see you back immediately if you’re getting worse. I’ll recheck you in a week.” The diagnosis (with associated ICD-10 codes of course) is automagically recorded, and the physician’s favorite dose of amoxicillin is queued up for his review and electronic signature.

This George Jetson future is near, at least it seems to be that way. Voice recognition vendors are today using natural language understanding (NLU) to pull concepts, orders, and discrete data from free-text notes. It’s not as smooth or easy as the cartoon concept proposed earlier, but the basics are in place today and beta testing is happening right now.

How does sound design fit into this rosy scenario? Imagine the not-too-distant future when a physician is interacting with the technology described above. How does the doctor know if the orders were heard clearly? How does she know if clarification or additional information is needed? There are no right or wrong answers to these questions, but the answers should be based on solid sound design principles. Is there a ping when an order is successfully and completely queued up? Is there a voice that asks for more information (“Doctor, will the hand x-ray be three-view or four-view?”) When should that extracorporeal voice ask its question? Perhaps immediately or perhaps it’s better to wait until the end of the office visit in the off chance that the doctor answers the question during the course of the ongoing conversation.

The reason sound is becoming increasingly important is due to the fact that doctors don’t want to spend most of their time looking at computer screens. (I know I’m not disclosing some unknown fact here so no need to roll those eyes.) Most physicians want to look at their patient when they’re talking to him/her. That’s human nature, of course, but it also gives hints that are important for the doctor to appreciate. Is the patient looking down and not making eye contact? Is the patient in obvious pain as the story is told? These are essential findings that are missed when typing on the keyboard or scanning the monitor.

As the Wired article notes, sound designers of the future likely want to avoid aural skeumorphism. It’s probably a bad idea to hear the cha-ching that a cash register makes (or used to make when people paid for stuff with cash) when the doctor successfully drops a charge or two. Don’t get me wrong; it’s important to alert the physician that charges have been dropped (no margin, no mission). But this notification should be subtle, yet obvious. It’s almost as if I’m talking about . . . usability for the ears.

I look forward to the day that audio and video work seamlessly to help clinicians do their jobs. We’ll know we’ve arrived when it seems like it’s always been the case that a computer is listening to us do our jobs and helps us (really helps us) make it easier and more efficient to get through the day.

Our Headquarters

Avaap USA, LLC
510 Thornall Street, Suite 250, Edison, NJ 08837

Phone: 732.710.3425
Fax: 732.243.9550
Email: info@avaap.com

Avaap Midwest 
6 West Fifth Street, 8th Floor, St Paul, MN 55102

Avaap Chicago 
70 W Madison Street, Suite 2610, Chicago, IL 60602

Global Center of Excellence Chennai
Chennai ONE, IT SEZ, Upper Stilt, Pallavarm-Thoraipakkam, 200 ft Road, Thoraipakkam, Chennai 600096. India

Global Center of Excellence Pune
Work Lab 2.0 5512, Ganeshkhind, Aundh, Pune 411007. India

Avaap UK
Stanmore House 64-68 Blackburn Street Radcliffe, Manchester M26 2JS. England

Avaap Spain
Antonio Machado 78, Australia Building 1-A3, Viladecans, Barcelona 08840 Spain

Avaap Amsterdam
Keizersgracht 62-64 Amsterdam, Netherlands 1015CS

Technical assistance for Avaap Managed Services and Product customers: support@avaap.com

Contact Us

Thank you for getting in touch. Let us know how we can help, and we'll get back to you ASAP!