AR-Tho
Project Details
| Course | Other Team Members | Language(s) |
|---|---|---|
| Launch Project (GIX) | Ross Todrzak, Shannon Truong, Yue Yu | C, C++, C# |
Project Touchstone
What are the consequences of a surgeon not receiving effective training?
Primary and Secondary Research
Over the course of 3 months, we investigated this by conducting both literary reviews and interviewing with multiple orthopaedic surgeons and other experts of the medical or augmented reality space. A 2008 study found that 63.5% of major complications are caused by surgical technique error. Our interviews, with 3 instructor surgeons and 2 residents, uncovered there was limited time for training and feedback, and a high need for multi-sensory input, such as information through touch and sight, for successful patient outcomes.
Current solutions don’t meet these needs. Textbooks, simulations and virtual reality provide one modality (sight), while cadavers require time, money, and space.
Unlike the purely digital virtual reality, AR is an interactive virtual layer on top of reality. A 2021 study by Microsoft Education Mixed Reality found that, when used in learning, there is a 22% improvement in test scores and a 35% increase in student engagement and retention.
Problem statement
How do we provide residents and medical students a solution in augmented reality that are more:
- Immersive
- Accessible
- Repeatable
- Multi-sensory
- Lower in-patient risk
Solution
Introducing AR-Tho….
A multi-sensory augmented reality training platform that overlays virtual anatomy onto 3D printed/silicone models. Users develop 2D to 3D spatial translation and fine motor control in a repeatable, immersive, and low-risk environment. A resident is able feel the difference when cutting through cartilage, and muscle as they’re seeing it in the virtual environment.
As a proof of concept, the digital block, or localized anesthesia of a finger, was chosen to demonstrate how our triangulated solution allows residents virtual x-ray-like information to enhance interactions with real world models. The finger is simple enough to test but complex enough to need repeated training. We envision this technology to be used to train for more complex procedures such as epidurals.
Process
Prototype Architecture
Competitive Analysis
Demo
Value Proposition
AR-Tho’s contribution to the industry - its value proposition – is its unique synthesis of existing platforms to create a system allowing for learning with both the tangible and virtual. Users can see the needle being inserted and feel if the needle has hit tissue, cartilage, or bone. It is multi-sensory training that textbooks and virtual reality can’t offer.
Our system’s tactile feedback is provided through silicone models, which are reusable. This opens training opportunities to be cost-effective and accessible, allowing users more hands-on training on their own schedule and location. AR-Tho is an expansive system that can be applied in multiple medical specialties with add-ons and packages in mind.
Next Steps
Our journey doesn’t stop here. We have planned out the next steps to improve AR-Tho and move it forward from the proof-of-concept stage. We will implement computer vision using third party software. We will also start developing a feedback component that gives a quantified measure of successful learning for instructors and students. Routes we’re exploring are augmentations to the silicone model that will provide instant feedback, such as haptic feedback or virtual feedback similar to video games. Lastly, we want to track tangible the surgical equipment so that users can see the tool as they’re using them.