Motif: Discovering and Automating Personal Web Workflows
2026-07-12 • Human-Computer Interaction
Human-Computer InteractionArtificial IntelligenceSoftware Engineering
AI summaryⓘ
The authors created a system called Motif that watches how people use their web browsers to find tasks they repeat often, which could be automated. Instead of users having to figure out what to automate, Motif suggests programs based on these patterns and lets users approve and tweak them using simple language. In a study, Motif found more useful automations than users could on their own, and most participants liked using the programs it created. This helps make automation easier and less expensive by working quietly in the background.
large language modelsprogramming by demonstrationautomationbrowser activitypattern recognitionnatural language programminguser studyvibe coding
Authors
Shaokang Jiang, Daye Nam
Abstract
Recent advances in LLMs and existing work on programming by demonstration have made it possible for end users to create automations by explicitly demonstrating their behavior to LLMs. However, these approaches rely on the assumption that users know what to automate and what is capable of being automated. Additionally, automation via LLM agents is often expensive compared with programs. We introduce Motif, a system that passively observes everyday browser activity to discover recurring interaction patterns that are programmable, makes recommendations to users whenever a pattern is discovered and generate a program to install after user confirmation. Users can review, and refine the program using natural language. We evaluated Motif in a multi-day study, comparing its ambient discoveries against automations users attempted to build via ``vibe coding.'' With eight participants, Motif discovered more automatable patterns than users recognized. Most of them matched participants' routines and were useful. Follow-up surveys showed most would continue using Motif-generated programs.