The short version
RunAnywhere's MetalRT is a new software engine by YC-backed startup RunAnywhere that runs AI models like chatbots, speech-to-text, and voice generation blazingly fast right on Apple Silicon chips in Macs and iPhones, without needing the internet. They've open-sourced RCLI, a free tool you can install in minutes to talk to AI using just your Mac's microphone and speakers – think Siri but smarter, snappier, and fully private. This beats competitors like Apple's own tools and others in speed tests, making on-device AI feel as quick as cloud services without the privacy risks or data costs.
What happened
Imagine you're chatting with an AI assistant on your Mac, speaking into the mic, and it listens, thinks, and replies out loud – all in a split second, no Wi-Fi needed. That's what Sanchit and Shubham from RunAnywhere just launched on Hacker News. They're YC W26 founders (a top startup accelerator) who built MetalRT, a custom engine tuned for Apple's M-series chips (like in recent Macs and iPhones). It powers three key AI tasks: large language models (LLMs, the tech behind chatty AIs), speech-to-text (STT, turning your voice into words), and text-to-speech (TTS, turning words back into voice).
They tested it head-to-head and won: MetalRT generates AI responses 1.67 times faster than llama.cpp (a popular open tool), 1.19 times faster than Apple's MLX, and crushes voice tasks – transcribing 70 seconds of audio in just 101 milliseconds (714 times real-time speed). They've open-sourced RCLI, a simple command-line tool (like a supercharged terminal app) that chains these together for full voice conversations. Install it with Homebrew (a free Mac package manager), download about 1GB of AI models once, and you're talking to AI locally. No cloud servers, no API fees, no sending your voice to strangers.
Why build this? Demos of local AI look cool, but real apps lag because voice AI stacks three slow steps: listen, think, speak. Add up the delays, and it feels clunky – like waiting for a sluggish friend to respond. RunAnywhere skipped bulky middle layers, writing custom "shaders" (think optimized recipes) directly for Apple's Metal graphics tech. Everything pre-loads in memory, so there's zero hiccups mid-chat. It's MIT-licensed (fully free to use and tweak), with demos on YouTube showing it handle 38 voice commands, search your files, and swap models on the fly.
Why should you care?
For everyday folks, this flips AI from "cool but glitchy offline toy" to "reliable sidekick." Cloud AI like ChatGPT or Siri feels instant because servers do the heavy lifting, but they listen to everything you say, charge per use, and die without internet. Local AI keeps your data on your device – private, no bills, works on planes or spotty Wi-Fi. RunAnywhere makes it fast enough to replace those, so your Mac becomes a privacy fortress with pro-level smarts.
Picture brainstorming ideas offline, transcribing meetings instantly, or getting voice replies while hiking – without Big Tech peeking. It's a big deal because on-device AI has been "almost there" for years; now it's here, beating Apple's own speed. As phones and laptops pack beefier chips, expect apps to ship with this baked in, making AI feel native, not bolted-on.
What changes for you
- Easy setup for tinkerers: If you have a recent Mac (M1 or newer), paste three commands into Terminal: tap the brew source, install rcli, run setup. Boom – push-to-talk AI chats. It's interactive, shows speeds per step, and falls back to other tools if needed.
- Privacy boost: No more "Is Siri spying?" worries. Your voice stays local; great for sensitive work, family talks, or health notes.
- Offline superpowers: Travel? Dead zones? Still get fast AI help – dictate notes (super-quick STT), generate speech, or query docs with RAG (retrieval-augmented generation, like AI searching your files in 4ms).
- Cheaper long-term: No subscription creep. Download models once (~1GB), run forever. Developers will build apps around this, so future Mac/iOS software gets snappier AI without phoning home.
- Voice feels natural: Latency under 600ms total means conversations flow like talking to a person, not a robot. Try the YouTube demo – it's eerie how smooth.
This isn't locked to pros; the open-source RCLI invites hobbyists to experiment, sparking cooler apps. RunAnywhere plans enterprise tools too, but the free version empowers you today.
Frequently Asked Questions
### Can I use this on my Mac right now?
Yes, if you have Apple Silicon (M1 or later Mac). Install via Homebrew with brew tap RunanywhereAI/rcli https://github.com/RunanywhereAI/RCLI.git, then brew install rcli and rcli setup. It downloads models once, then you're in interactive voice mode with push-to-talk. No cloud needed.
### Is it really faster than Apple's own AI tools?
In their tests on an M4 Max Mac, yes – MetalRT hit 658 "tokens per second" (think words generated per second) on small models, beating Apple's MLX by 19% and others by up to 2x. Voice pipelines are 4-7x real-time fast, chaining listen-think-speak without lag.
### Does this work on iPhone or only Macs?
The launch focuses on Macs via RCLI, using Apple Silicon like M-series chips. RunAnywhere's full platform supports iOS and Android for apps, but check their site for mobile demos – it's cross-platform under the hood.
### Is it free and safe? What about my data?
Totally free (MIT license) and safe – everything runs on-device, no internet pings or API keys. Models download from trusted sources during setup; your mic input never leaves your Mac.
### What can I build or do with it once installed?
Chat by voice, control 38 Mac actions (like opening apps), search your files instantly, or swap 20+ models. It's a full pipeline for voice AI experiments – perfect for offline assistants, note-taking, or custom bots.
The bottom line
RunAnywhere's launch of MetalRT and open-source RCLI means fast, private AI voice chats are now a reality on your Apple devices, outpacing even Apple's tools without needing the cloud. For regular people, this unlocks reliable offline smarts – think private Siri 2.0 for travel, work, or play – saving money, boosting privacy, and making tech feel magical. Download it today if you're on a modern Mac; it's the first taste of AI that truly runs "anywhere" as fast as promised. Watch for apps adopting this – your future iPhone might never need cloud AI again.