Tobias Weinberg (left) at Cornell Tech with his adviser, Thijs Roumen. (Alexandra Bayer / Cornell University) Students PhD Student Aims to Help Others with Speech Disabilities Stories You May Like Why I’ve Become a Champion for Disability Equity in Academia Undergrad’s Start-Up Lets You ‘Hear’ through its Eyeglasses Island of Digital Dreams: Fascinating Facts About Cornell Tech Tobias Weinberg is unable to talk; his research taps AI to pepper written conversations with asides, humor, and even sarcasm Editor’s note: This story has been adapted from a feature in the Cornell Chronicle. By Caitlin Hayes At 15, when a neurological condition took Tobias Weinberg’s ability to speak, aspects of his personality became more difficult to express. Typing to communicate, he struggled to keep up in conversations, especially to make the jokes or sarcastic comments that had been his norm. And his first text-to-voice device was monotone, with Mexican or Spanish accents but not his native Argentinian. “The monotone voices, the timing of interjections, and conveying my personality through this new way of communication was definitely frustrating,” writes Weinberg, now a doctoral student and Siegel PiTech Fellow at Cornell Tech. As part of the Matter of Tech Lab, he is exploring how artificial intelligence can enhance the technologies that he and more than two million Americans with speech disabilities use to communicate. Weinberg communicates via a text app on his phone. Through a standing partnership between Cornell Tech and YAI—a nonprofit that supports more than 20,000 people with intellectual and developmental disabilities in New York, New Jersey, and California—Weinberg spent a year working with a group of augmentative and alternative communication (AAC) users who live in group homes in Tarrytown, NY, to better understand needs and behaviors and to improve prototypes. The resulting research and lines of inquiry, which incorporate Weinberg’s own experience, could transform assistive technology design. The field is taking notice. Weinberg’s first paper—Why So Serious?: Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces—won the best paper honorable mention and jury best demo award at the prestigious Conference on Human Factors in Computing Systems. Weinberg is exploring how artificial intelligence can enhance the technologies that he and more than two million Americans with speech disabilities use to communicate. Those are highly coveted commendations, according to his adviser, Thijs Roumen, an assistant professor at Cornell Tech. “Tobi really is a trailblazer,” says Roumen, who has a joint appointment in the Bowers College. “He’s been developing technology while also using the technology, which changes the way it’s shaped and the way we reflect on it. In the process, he’s hitting on a richness that is going to make the future of AAC technology much, much better, and he’s also inspiring a whole generation of researchers.” Heather Klippel, who has cerebral palsy and lives in a YAI group home, has similar frustrations with AAC devices to Weinberg’s: she gets overwhelmed when too many people are speaking and struggles to convey tone and humor. “Those things are very hard to express as a nonverbal person,” Klippel writes. Chatting with a fellow doctoral student. In the first of two studies, Weinberg interviewed Klippel and six others and designed an interface that could help users write jokes or humorous comments they can then interject in real time. “There is an inherent tradeoff between agency and efficiency when designing AI tools that support communication,” Weinberg writes. “While an AI auto-complete will enable making humorous comments faster, there is a risk that it diminishes the user’s sense of agency by making jokes for users instead of with the user.” Weinberg designed interfaces that explored this tradeoff. In one, users selected keywords they wanted the AI to use in crafting a joke; in another, they were able to edit and modify AI-written jokes; and in another, they could simply choose a joke that the AI provided. Tobi really is a trailblazer. Prof. Thijs Roumen Stories You May Like Why I’ve Become a Champion for Disability Equity in Academia Undergrad’s Start-Up Lets You ‘Hear’ through its Eyeglasses “What we found is in time-pressured scenarios, like making a humorous comment, AAC users were willing to give up some agency to deliver the comment faster,” Weinberg writes. “This challenged the existing research that said AAC users care most about maximum agency, which is true in general but not always.” That led Weinberg and his collaborators to think about the purpose of humor. Often, he says, the joke itself is less important than participation and engagement in the conversation. The team started to consider other types of “backchanneling,” or ways we communicate engagement, alongside the primary conversation, like saying “uh-huh” or nodding. Weinberg wants to make it easier to have conversations and make jokes. A second study with the AAC users resulted in a paper, One Does Not Simply ‘Mm-hmm’: Exploring Backchanneling in the AAC Micro-Culture, that was presented at a conference in October 2025. Weinberg and his team found that the participants formed their own micro-culture of backchanneling, such as tapping their armrests to indicate agreement or raising eyebrows. The interviews and observations led him and his team to recommend a design approach that amplifies and incorporates what users are already doing, rather than imposing mainstream behaviors. Thinking about the purpose of humor, Weinberg finds that a joke itself is often less important than participation and engagement in the conversation. “There can be this tendency to just want to build an app and solve a problem,” Roumen says. “But by asking ourselves these fundamental questions and driving the curiosity that Tobi brings as a researcher to really understand what’s happening, we can now start to understand how we can be really impactful in this space.” Those fundamental questions are often also ethical ones. For a third paper currently in submission, Weinberg developed an app that collected everything he’d typed over a period of seven months and used the text to train a large language model that could help facilitate and speed his communication. While the resulting “AI-twin” captured a verbal identity—incorporating characteristic phrases and Argentinian slang—it failed in practice to suggest or provide that language in appropriate contexts and risked exposing private information at the wrong times. Weinberg also felt the app dampened control over his own self-presentation. Weinberg has found that expressing individuality, like using slang, can be challenging. “AI is a very wonderful but dangerous technology, especially if it mediates everything we say as AAC users,” Weinberg writes. “So, my work serves both sides, providing design guidelines for future developers and also playing the villain, warning of the socio-technical implications of AI in the lives of AAC users like myself.” Weinberg disassembled his first computer at age two, and at seven told his parents he wanted to invent things that would help people. But when he arrived at Cornell Tech for a summer internship in 2022, he didn’t know what a PhD was and did not see it in his future. Wendy Ju, an associate professor at Cornell Tech, encouraged Weinberg to apply for the doctoral program after completing his bachelor’s in mechanical engineering at the Technion–Israel Institute of Technology. AI is a very wonderful but dangerous technology, especially if it mediates everything we say as AAC users. In 2023, he joined Roumen’s lab, intending to work on digital fabrication. But Roumen encouraged Weinberg, as he does all students, to find a project he really cared about. “I told Thijs, there was this other thing I really care about, but neither of us has any experience with it,” Weinberg writes. “He was on board to give it a try, and here we are.” Looking ahead, Weinberg hopes to reframe AAC—not as a workaround for missing speech, but as a medium of expression. “This vision represents a step toward the broader goal of enabling AAC users to fully participate in spoken communication,” he writes, “and to flourish in society.” Top: Tobias Weinberg (left) at Cornell Tech with his advisor, Thijs Roumen. (All photos by Alexandra Bayer / Cornell University) Published November 6, 2025 Leave a Comment Cancel replyOnce your comment is approved, your email address will not be published. Required fields are marked *Comment * Name * Class Year Email * Save my name, email, and class year in this browser for the next time I comment. Δ Other stories You may like Ask the Expert Rick Geddes on the State of U.S. Infrastructure—and the Evils of Deferred Maintenance Alumni ‘Sheer Awe’: Recalling the Legendary Grateful Dead Concert of May ’77 Cornelliana Gothic Castle on the Hill: Fascinating Facts about Risley Hall