Work

2025 to present · co-founder and CTO

FluenTea: a video classroom with an AI speaking coach

I co-founded FluenTea so that language schools can help their students actually speak a new language, and prove that they do. Teachers run live classes in it. An AI coach called ChaCha sits in, runs practice calls between classes, and turns what students say into feedback for the student, the teacher and the department. I was the technical co-founder building the product.

  • Concept to launch took three months. ChaCha has only grown since then. She answers questions, and she acts on your behalf too: creating assignments, grading them and finding you a language partner.
  • Safety and testing. Every action the coach can take is listed in a manifest that tests enforce, and anything that reaches another person needs a confirmed second turn. A red-team suite feeds her hostile text, and a second model grades her real answers against a baseline per error type that each release has to match or beat.
  • Listening to users. We found that users are more candid talking to ChaCha than to the team, so we set her up to run the interviews. Many of our UX fixes and upgrades came from what we learned there.
  • Finding problems before anyone reports them. A nightly script signs up and uses the app as a confused first-time user and files what breaks. Microsoft Clarity shows where people rage-click, ChaCha's transcripts double as a complaint channel, and the team holds its own meetings inside FluenTea.

Launched three months after the concept. More than 16,000 registered users from 136 countries, and three universities as clients.

Python · FastAPI · PostgreSQL · React · LiveKit · Kubernetes on GKE · Cloud SQL · Cloudflare · Capacitor · Stripe

The product tour, two minutes (October 2026).
Click a screenshot to enlarge it.
2021 to 2026 · applied scientist, Amazon Berlin

Computer vision and multimodal models on the retail site

In a Berlin research team that partnered with product teams across the company, I was technical lead on three projects that went to production in front of customers, with multi-million dollar impact from higher sales and fewer returns. Five papers at Amazon's internal computer vision conference, one of them shortlisted for best paper.

  • What is in a creator's photo or video (2023 to 2024). I benchmarked zero-shot and low-shot CLIP against a supervised baseline and designed the crowd annotation over 270 interests and 30,000 videos. The model launched into Amazon's interest mapping and feeds two recommenders. Lead author of the internal paper, an oral and best-paper shortlist.
  • Generative video and 3D (2025). Investigated frame selection for structure-from-motion and Gaussian splatting from 3D object captures, and made product videos with diffusion and image-editing models for video transitions and product swaps.
  • Product sizes from images (2021 to 2023). I proposed treating it as text visual question answering, built the golden dataset with a five-person annotation team across 130 product types, productionized the model to backfill more than ten million products, and led the public dataset release. Door Desk award; D-Extract at WACV 2023.
  • Visual similarity for Alexa Shop the Look (2022 to 2023). Embedding retrieval with diversity filters.
  • Also at Amazon. Supervised four interns and interviewed for the team (17 loops in one year), organized the Berlin ML Workshop 2023 with 180 attendees, won a GenAI hackathon, and received the Amazon Berlin Champion of Diversity and Inclusion award.

PyTorch · CLIP and vision-language models · text VQA · embedding retrieval · diffusion models · crowd annotation · A/B testing

Architecture figure from D-Extract: product image split into patches and OCR text boxes, fed together into a transformer with an image classification head and a bounding-box classification head
From D-Extract (WACV 2023): image patches and OCR boxes go through one transformer to find the dimensions. Paper · Dataset.
2019 to 2021 · research staff member, IBM Research Almaden

Table extraction for enterprise documents

Most of the numbers in a business document sit in tables, and in 2019 machines were still bad at reading them. At IBM I worked on the models and on the tools that let a domain expert correct them and train their own.

  • GTE (Global Table Extractor): technical lead and main developer of an end-to-end system that finds tables on a page and recovers their cell structure using visual context. WACV 2021.
  • TableLab: a labeling interface on top of GTE, from concept to launch, where users correct detected tables and the corrections retrain the model for their documents.
  • SEM-TAB-FACTS: lead organizer of SemEval 2021 Task 9, fact verification over tables in scientific papers. Co-author of CORD-19 (ACL 2020 workshop) and of D2S, document-to-slide generation (NAACL 2021).
Framework figure from the GTE paper: table detection with a penalty loss and re-ranking, then a cell network chosen by whether the table has graphical lines
From GTE (WACV 2021): table detection and cell structure recognition in one framework. Paper.
TableLab interface: a scanned financial report with two detected tables outlined in green, and the recovered cell structure on the right with merge, split and move controls
TableLab: detected tables on the page, recovered structure on the right, ready to correct.
2014 to 2018 · PhD, University of Washington

Decoding behavior from brains and video

My thesis used machine learning to understand the brain, and the brain to improve machine learning. The data was hundreds of hours of hospital recordings that would otherwise have been discarded: patients with electrodes on the surface of the brain, alongside video and audio of whatever they happened to be doing in the room. From it we decoded their natural behavior from brain activity.

  • AJILE (Annotated Joints in Long-term ECoG): sensor-fusing deep learning models that decode and predict movement from neural recordings and video, in natural rather than experimental conditions. Oral at AAAI 2018.
  • Automated annotation. Founded the project to annotate naturalistic neuroscience data automatically with computer vision and audio models. It became a multi-year effort funded by NSF and DARPA, and parts of the dataset were released publicly.
  • Earlier. Decoding dreams from fMRI, automatic classification of fish behavior, and brain surgery in mice to study PTSD, during undergraduate research and a DAAD scholarship in Germany.

The released dataset · Papers on Google Scholar

Model figure from AJILE: at each time step before a movement, a video frame and an ECoG window pass through separate CNNs, are fused, and an LSTM predicts the coming movement
From AJILE (AAAI 2018): video and brain recordings fused to predict a movement before it starts. Paper.