Study: A11yShape: AI-Assisted 3-D Modeling for Blind and Low-Vision Programmers (DOI: 10.48550/arXiv.2508.03852)
Blind and low-vision programmers have long been locked out of three-dimensional modeling software, which depends on sighted users dragging, rotating and inspecting shapes on screen.
Now, a multiuniversity research team has developed A11yShape, a new tool designed to help blind and low-vision programmers independently create, inspect and refine three-dimensional models.
The team consists of Anhong Guo, assistant professor of electrical engineering and computer science at the University of Michigan, and researchers from the University of Texas at Dallas, University of Washington, Purdue University and several partner institutions-including Gene S-H Kim of Stanford University, a member of the blind and low-vision community.
A11yShape combines the code-based 3D modeling editor OpenSCAD with the large language model GPT-4o. OpenSCAD already lets users create 3D shapes by writing commands instead of dragging objects with a mouse. For example: cylinder(h=20, d=5) instantly produces a 20-unit-high cylinder 5 units in diameter that can be rotated and exported for 3D printing.
But for blind users, that rendered model is just a silent picture on the screen. They can write code but have no way to "see" whether the shape is tall or short, where parts are placed or if something is misaligned.
A11yShape fills that gap by acting as the user's "eyes." Every time a blind or low-vision user writes OpenSCAD code, A11yShape:
- Renders the 3D model from several angles-top, bottom, left, right, front and back-so there is a complete visual snapshot of the object.
- Feeds both the code and those snapshots into GPT-4o. Because GPT-4o can process text and images at once, it produces plain-language descriptions of each part's size, shape and position ("A tall narrow cylinder stands upright in the center attached to a cube at its base") and can answer questions ("How wide is the base?") or suggest code changes.
- Synchronizes selections across all views. Through a "cross-representation highlighting" mechanism, when a user selects (via keyboard or screen reader) "wing" in the outline, A11yShape simultaneously highlights the wing's code, its description and its rendered position. The system also records every change both user edits and AI-suggested edits-in separate logs so users can backtrack or compare versions.
Together, these features give blind and low-vision programmers four linked ways of understanding a model-code, AI-generated description, semantic hierarchy and visual rendering-letting them build and adjust designs independently.
To evaluate A11yShape, the team ran a multisession study with four blind or low-vision programmers, none of whom had prior experience with 3D modeling. After an introductory tutorial, each participant used the system over three sessions to complete 12 models, including a Tanghulu skewer, robots, a rocket and a helicopter.
All four participants successfully completed both guided and free-form 3D modeling tasks using A11yShape. They reported a mean System Usability Scale score of 80.6, a high mark for usability. One participant said, "I had never modeled before and never thought I could. … It provided us (the BLV community) with a new perspective on 3D modeling, demonstrating that we can indeed create relatively simple structures."
The study revealed distinct workflows. Some participants wrote most of the code themselves, using the AI mainly for descriptions. Others relied on the AI to generate an initial model and then refined it manually. All used the version control and hierarchical navigation to correct mistakes and locate parts of their models.
Still, challenges remained. Heavy textual descriptions sometimes created cognitive overload. Participants also struggled to judge precise spatial relationships without tactile feedback, leading to occasional misalignments such as propellers floating above a helicopter's fuselage.
Despite its limitations, the researchers argue that A11yShape represents a milestone in accessible creativity tools. By linking code, descriptions, structure and rendering, the system lets blind and low-vision users independently design and edit artifacts that were once only accessible with sighted assistance.
Future versions may include more concise AI descriptions, auto-completion features for code and integration with tactile displays or 3D printing to provide physical feedback.
"Our vision for A11yShape is to open a door for blind and low-vision creators to step into a world of creative activities, such as 3D modeling, and to make what once seemed impossible, possible," said Liang He, a researcher at the University of Texas at Dallas.
"We're just at the beginning," Guo said. "Our hope is that this approach will not only make 3D modeling more accessible but inspire similar designs across other creative domains."
