Watson



 SYNOPSIS

This was my work conducted during my time as an HCI student researcher for the Vanderbilt University Medical center.  Building on the data and evidence of research…[in] Phase 1, IBM and VUMC will conduct a User-Centered Design Evaluation Study of IBM Micromedex: Phase 2 (the “Study”). This study will assess the performance and engagement of different Micromedex user groups at VUMC.  

 

 

TYPE

UX/HCI Research

 

 TOOL(S)

UX Research

IBM-Watson-Overview.001

 

The Core Experience 

Watson is a support system that incorporates a NICE accredited pharmaceutical database to enable clinicians to deliver insights for better clinical care
Watson is integrated to gather search intents through conversation, provide and guide specific clinical solutions*
The goal here is to obtain information through a “human-like” way, to ask questions the way we would ask them in real life and receive the answers as if we were talking to someone real. This program is accessed through a chat window. 

 

Our Goals
  • Refine (and redesign) Micromedex User Interface 
  •  Inform further targeting of the tool for diverse purposes
    Training, Clinical Care, Pharmacy (etc.)  
  • Contribute to generalizable knowledge of the user and organizational workflow practices
  • Inform future technological development  
Aim 1: Assess the accuracy and clinical utility of Micromedex Answers 

Conduct virtual interviews (individual, group) with a diverse group of study participants to explore how Micromedex is used 

A subset will be asked to keep a journal containing: 

  • Questions regarding medication management (~200 questions) 
  • Records associated with query terms
    • what answer they found
    • where they found it
    • whether they knew the result  
Aim 2: Assess the usability of Micromedex and Micromedex with Watson Assistant interface

Recruit diverse sample of clinical staff, health care providers that consist of a relatively even distribution of physicians, nurses, and pharmacists to participate in a usability study. They will be drawn equally from Micromedex users and/or Micromedex and Watson users and non-users 

Perform usability evaluation with Micromedex and Micromedex with Watson Assistant to determine efficiency, effectiveness, and satisfaction with the system. 

Untitled
Homepage
  • 2 separate choices for lookup might lead to user-confusion during selection
  • Redundancy of “Ask Watson” button on the page might lead to more clutter and confusion
  • The Ask Watson button at the bottom doesn’t have a very strong “call to action”
  • The Latest News, Support & Training, and Resources widgets are bigger than the actual interface lookup, leading to a bottom-heavy visual hierarchy 
 
Untitled-2
Drug Interactions/Drug Comparison 
  • The user input box for matching drug names is difficult to understand - much easier to go with a forgiving format input instead of exhibiting all the drug names on the side 
  • Instead of having a box for drugs to check, better off holding that in a component with names that dynamically get added when the user adds inputs  
  • Can have a separate component for adding allergens, the association with “Drugs to check” is potentially confusing 
Untitled-1
IV Compatibility 
  • The labels of the “Drugs” and “Solutions” could serve to be more accentuated 
Untitled-2-1
Tox & Drug Product Lookup
  • The description of what this page does doesn’t have to be open all the time - this breaks the pattern of the “information” button that we see on the other pages 
  • Instead of having “I want to search for…” on the side, it would be more beneficial to follow UI/UX design patterns and have these as specifications under the search bar 
  • This would reduce clutter and be more cohesive to user flow 
Untitled-3
Watson Assistant
  • For complicated drug names, no way to have an auto-correct or flexible search if there has been a misspelling 

  • User has to type in every step instead of providing options for further ease of lookup. 

  • The same logic can be applied to the “clear ____” option. It would be easier to have a button like “is this correct?” Or “Let me put in another option” following the bot response 

  • No indication of feedback when the lookup takes awhile

Usability Testing Results: 

The immediate details are confidential, but SUS testing was conducted. More than 60 medical practitoners were surveyed and interviewed. Based on patterns we saw in the interviews, suggestions were made to IBM regarding the redesign and update of Watson. 

"When making a query, what prompted you to use  <the method of search that the participant used>? (Ie. What made you want to use the search engine? Why did you go through this channel/tab?)"
"(For Watson) Did you notice that there was any other way to make this query? (I.e. Did they notice that Watson was a search option on the website interface?)" 
"Are there any tools or capabilities of Micromedex that you didn’t know existed before looking at the help/support section?""What information gathering capability of Micromedex do you use that you feel like is better/more unique that other medical databases? (What elements of Micromedex with Watson are our users looking for specifically when they open the webpage?)"
"What are some advantages that other medical databases have over Micromedex? (IBM might not like this? But could be useful!)" 

 

View