A screenshot from a live prototype after machine learning assigns the Veteran a disability
Automating Disability Claims on VA.gov
DESIGN FOR AI, LEAN UX, USER TESTING
The VA disability claims process is confusing, lengthy, and convoluted—but with a new algorithm crafted by VA Presidential Innovation Fellows, that could change.
Framing the problem
For Veterans and their caretakers, the disability compensation process can take years. It’s stressful. There’s lots of paperwork. But in 2019, VA.gov launched the online disability claim form and saw claims jumping in just a few weeks. Now that VA had more digitized claims to process, the feature proving successful, the UX team asked…how can we make this form even easier for Veterans to complete?
Why this project?
We heard about a small team of Presidential Innovation Fellows using machine learning to improve the disability claims process. Two engineers had trained an algorithm on data from over 5 million previous claims from Veterans. The algorithm took Veterans’ text, parsed it into VA official disability claim terms, and assigned a disability type to submit with the claim. This discovery peaked our interest—we’d heard in user research that Veterans sometimes agonize over choosing the right disability to claim—so we reached out to form a collaboration with the AI team.
Potential user flows for adding a new disability to claim using the automation feature
There were potential risks, though—like what if the machine model assigned the wrong disability to a Veteran’s claim? What if a Veteran’s text was too general to properly assign any disability? What was the risk for that Veteran’s claim if they submitted the wrong disability? Mostly importantly, would a Veteran trust an algorithm to assign them a disability based on their own text? We needed trust from the Veteran first.
An excerpt from a whiteboarding session, where we defined potential “gain” points and pain points for machine-learning disability claims
The AI team assured us that the model’s accuracy (over 80%) was worth exploring and we agreed. Anything that lessens tension for the Veteran and shortens the claim process (which can already take months or sometimes years) would be a win. Veterans deserve access to their well-earned benefits and the UX team was driven to make this experience easier, more accessible.
We only had a few weeks to create a working prototype that using the new disability-assignment algorithm, so we took a lean UX approach by providing just enough design guidance and information for the developers to quickly craft the prototype and deploy it to a staging environment. Some nimble QAing and tweaks later—we had a working simulation of the automated disability assignment!
With a functional prototype, we took to user testing. I had some assumptions and questions — would Veterans and caregivers like having an algorithm assign them a disability to claim automatically? How much of the process would they want to see? Would they feel comfortable with something so important be up to a computer? It’s true that algorithms decide many things for us now, from jail sentences to medical procedures to what we should watch next. We set out to see whether Veterans would trust a computer with this sensitive task.
The risk paid off. Surprisingly, Veterans trusted the automation to choose the right disability assignment more than they trusted themselves. It removed a large source of tension from the user flow—many Veterans would pause, stressing the select the correct disability from a lengthy list—and allowed them to complete this workflow with much more ease.
Conclusion
This project was a lean UX and development success. The downside that my team only had a month left on the contract ended up also being an upside, as the time constraint motivated us to work light, quick, and smart. We went from concept to testable prototype and saw encouraging results, that AI can be used for good to give Veterans the security of a VA-approved answer for which disability to claim.
📝 Please note that this project was completed at the end of a contract. My team handed off our designs and learnings, but we weren’t able to see it through to a full release.
If could do it again, I would want additional time to see the project through to a full release, because the results of our experiment were so promising and my fellow team members were a pleasure to work with.