Polso - note voice to action

Speak a note into a smartwatch and it becomes a calendar event, a note or a shopping-list item on the phone — always with a human approval step, for an estimated €0.11 a month.

Category
AI & Automation
Year
2026
Role
Concept, agent design and build
Stack
Android (Pixel 9 Pro) · Galaxy Watch 8 · Whisper (on-device) · Fal.ai · Gemini / Claude / GPT (switchable)
Polso - note voice to action

01

Problem

Quick ideas and errands get lost because capturing them on a phone takes too many taps. Voice memos pile up but never turn into actions.

02

Approach

Notes recorded on a Samsung Galaxy Watch 8 reach a Google Pixel 9 Pro through Android's built-in sync and are transcribed with Whisper, which runs directly on the phone: fast, free and private, with excellent English and good Italian results. An LLM, guided by a system prompt with a fixed set of allowed actions, picks the right one: add a calendar event, create a note or list, or add items to the shopping list.

The agent never acts on its own. When a note is ambiguous it asks a follow-up question in text, and every proposed action must be approved one by one before it runs.

The model is not hard-wired. Gemini is the default, but the settings accept any Fal.ai model endpoint, with one-tap presets for Claude Sonnet 4.5, GPT-4o mini and Gemini 2.5 Flash, so models can be compared on real notes.

03

Result

Built in October 2026 and used daily since: 2–3 notes a day, with an estimated compute cost of about €0.11 a month, since transcription runs on the phone and only short text reaches the LLM. Ideas and errands now land in the right app in seconds instead of sitting in a voice-memo list.

Design decisions

  • A closed set of actions. The model chooses from a defined list instead of improvising, which keeps behaviour predictable and easy to test.
  • Human-in-the-loop by default. Nothing reaches the calendar or a list without explicit approval; ambiguity triggers a question, not a guess.
  • On-device where possible. Speech-to-text runs locally, so audio never leaves the phone and cloud calls stay tiny and cheap.
  • Model-agnostic. Swapping the model is a settings change, not a code change, so cost and quality can be balanced as models evolve.
Polso - note voice to action

Next project

Bento