The best Hacker News stories from Show from the past day
Latest posts:
Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
Hi HN,<p>I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal.<p>I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory.<p>The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are included.<p>The trick is to keep the shared part of the model and the KV cache in RAM, then stream only the routed experts needed for each token from SSD. An SSD is way slower than RAM, so the runtime uses a small expert cache and bounded parallel `pread`. While those reads are in flight, the GPU runs the shared part of the layer.<p>I ran more than 100 experiments. Most didn’t work. A few got me here. The experiments are described in the GitHub repo.<p>It currently generates 5–6 tok/s on an 8 GB M2 MacBook Air and 31–35 tok/s on an M5 MacBook Pro.<p>I also added an experimental OpenAI-compatible local server. It supports streaming and tool calls, and reuses one prompt prefix from the KV cache.<p>Try it! The Mac app is easy to install. On the first run, it will download 15 GB of weights from Hugging Face. The model is surprisingly capable.<p>I would love any kind of feedback!
Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac
Hi HN,<p>I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal.<p>I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory.<p>The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are included.<p>The trick is to keep the shared part of the model and the KV cache in RAM, then stream only the routed experts needed for each token from SSD. An SSD is way slower than RAM, so the runtime uses a small expert cache and bounded parallel `pread`. While those reads are in flight, the GPU runs the shared part of the layer.<p>I ran more than 100 experiments. Most didn’t work. A few got me here. The experiments are described in the GitHub repo.<p>It currently generates 5–6 tok/s on an 8 GB M2 MacBook Air and 31–35 tok/s on an M5 MacBook Pro.<p>I also added an experimental OpenAI-compatible local server. It supports streaming and tool calls, and reuses one prompt prefix from the KV cache.<p>Try it! The Mac app is easy to install. On the first run, it will download 15 GB of weights from Hugging Face. The model is surprisingly capable.<p>I would love any kind of feedback!
Show HN: How far do I have to go to run into 100k people?
Show HN: Scala Tutorials – interactive Scala 3 lessons in the browser
Show HN: Scala Tutorials – interactive Scala 3 lessons in the browser
Show HN: tale.fyi, we deserve a home for fiction
for decades, i have been concerned that the internet was being built around non-fiction, so i built something to show how we could celebrate great fiction on the web.<p>i started with an amazing library from the public domain, and i also added tools to add your own stories<p>super interested to hear any feedback, and if you read anything good!
Show HN: tale.fyi, we deserve a home for fiction
for decades, i have been concerned that the internet was being built around non-fiction, so i built something to show how we could celebrate great fiction on the web.<p>i started with an amazing library from the public domain, and i also added tools to add your own stories<p>super interested to hear any feedback, and if you read anything good!
Show HN: Yap – OSS on-device voice dictation for macOS with no model to download
Hey HN! I wanted to share this OSS project I've been working on.<p>It's called Yap and its a small menu-bar app for macOS that does voice to text for any input. You'll set a hotkey, press it, talk, press it again, and the text gets pasted into whatever field you were in. Everything runs locally and never leaves your computer. Fully OSS and MIT licensed.<p>With macOS 26, Apple recently added two new APIs, SpeechAnalyzer and SpeechTranscriber, that do streaming on-device speech to text using models the OS ships and manages. So the app ships no model of its own and loads nothing before the first word. A recent benchmark put Apple's model slightly ahead of Whisper Small on accuracy and about 3x faster (see: <a href="https://news.ycombinator.com/item?id=48894752">https://news.ycombinator.com/item?id=48894752</a>). On Mac, there's really no need anymore to download models or pay for expensive APIs.<p>A lot of existing dictation tools do one of a few things I wanted to avoid with this OSS project. They either:<p>- cost money (for something that's literally built into the OS)<p>- bundle memory-intensive models (e.g. Whisper or Parakeet)<p>- webapps wrapped in Electron<p>- Intel macs straight up don't work<p>- closed source<p>- use third-party APIs that will have access to all your transcripts<p>It's around 3,000 lines of native Swift in a 4 MB app and idles near 60 MB of memory. Audio comes off AVAudioEngine into SpeechAnalyzer with volatile results turned on for the live preview, history is stored in SwiftData. There's no network code in it at all.<p>Repo and a demo available here: <a href="https://github.com/FrigadeHQ/yap" rel="nofollow">https://github.com/FrigadeHQ/yap</a><p>Happy to answer questions and would love to hear any feature requests!
Show HN: I was tired of opening 2 tabs for every HN link, so I made a userscript
HN is great for the links people share, but a big part of the value I get comes from reading the discussion around them. I realized I was always opening the article in one tab and the comments in another, constantly switching back and forth.<p>I figured there was probably a simpler way, so I threw together this userscript to merge the two.<p>1. Clicking a link from Hacker News opens the article with a side panel containing the discussion. It doesn't require your credentials, is resizable, and is easy to tweak if you want to customize it.<p>2. If you land on an article that has previously been shared on HN, the script finds the existing discussion and adds a button in the top-right to open the panel.<p>Feedback welcome.
Show HN: I was tired of opening 2 tabs for every HN link, so I made a userscript
HN is great for the links people share, but a big part of the value I get comes from reading the discussion around them. I realized I was always opening the article in one tab and the comments in another, constantly switching back and forth.<p>I figured there was probably a simpler way, so I threw together this userscript to merge the two.<p>1. Clicking a link from Hacker News opens the article with a side panel containing the discussion. It doesn't require your credentials, is resizable, and is easy to tweak if you want to customize it.<p>2. If you land on an article that has previously been shared on HN, the script finds the existing discussion and adds a button in the top-right to open the panel.<p>Feedback welcome.
Show HN: I was tired of opening 2 tabs for every HN link, so I made a userscript
HN is great for the links people share, but a big part of the value I get comes from reading the discussion around them. I realized I was always opening the article in one tab and the comments in another, constantly switching back and forth.<p>I figured there was probably a simpler way, so I threw together this userscript to merge the two.<p>1. Clicking a link from Hacker News opens the article with a side panel containing the discussion. It doesn't require your credentials, is resizable, and is easy to tweak if you want to customize it.<p>2. If you land on an article that has previously been shared on HN, the script finds the existing discussion and adds a button in the top-right to open the panel.<p>Feedback welcome.
Show HN: Formally verified 3D CSG: Trust 93 lines spec, not 1000 lines AI code
To my knowledge, this is the first formally verified implementation of a 3D constructive solid geometry (CSG) operation: mesh intersection, implemented in Lean 4 and verified against a concise specification that pins down the surface of the resulting mesh exactly and guarantees practical well-formedness conditions on the triangulation.<p>This project is also an experiment in avoiding having to trust AI-generated code. A human reviewer only needs to read 93 lines of formal specification and run the Lean checker to certify the correctness of the kernel, skipping the intricate 1000+ lines of AI-written implementation. To prove correctness, AI autonomously wrote over 60,000 lines of Lean proofs, which also never have to be inspected by a human. The Lean checker guarantees conformance to the specification at compile time, with zero trust placed in any LLM. This allows us to treat the implementation and proofs as a black box. I guided the agent through the milestones described in the readme to arrive at the result presented here.<p>Also take a look at the web demo <a href="https://schildep.github.io/verified-3d-mesh-intersection/" rel="nofollow">https://schildep.github.io/verified-3d-mesh-intersection/</a>, which runs the verified mesh intersection kernel compiled to WebAssembly in your browser.
Show HN: Formally verified 3D CSG: Trust 93 lines spec, not 1000 lines AI code
To my knowledge, this is the first formally verified implementation of a 3D constructive solid geometry (CSG) operation: mesh intersection, implemented in Lean 4 and verified against a concise specification that pins down the surface of the resulting mesh exactly and guarantees practical well-formedness conditions on the triangulation.<p>This project is also an experiment in avoiding having to trust AI-generated code. A human reviewer only needs to read 93 lines of formal specification and run the Lean checker to certify the correctness of the kernel, skipping the intricate 1000+ lines of AI-written implementation. To prove correctness, AI autonomously wrote over 60,000 lines of Lean proofs, which also never have to be inspected by a human. The Lean checker guarantees conformance to the specification at compile time, with zero trust placed in any LLM. This allows us to treat the implementation and proofs as a black box. I guided the agent through the milestones described in the readme to arrive at the result presented here.<p>Also take a look at the web demo <a href="https://schildep.github.io/verified-3d-mesh-intersection/" rel="nofollow">https://schildep.github.io/verified-3d-mesh-intersection/</a>, which runs the verified mesh intersection kernel compiled to WebAssembly in your browser.
Show HN: XY – A Fast, composable, GPU-accelerated interactive plotting library
Show HN: XY – A Fast, composable, GPU-accelerated interactive plotting library
Show HN: Watch 14-Byte AI "brains" attempt to solve a 2D maze (Its hard)
Hey HackerNews,<p>I built this project over the last few weeks as a palette cleanser from a failed game launch.<p>I wanted to learn a bit about AI/Neural-Networks and naively thought I could build a tiny maze-solving AI in a weekend with a 100% solve rate.<p>Well - I couldn't, but I got pretty close. 14 Bytes total model size, and a 96.5% solve rate on unseen mazes. Trained across 46 phases experimenting with different ideas to improve the model (better performance, smaller size).<p>Its quite fun to watch the model attempt to solve the maze, when they fail its usually due to getting stuck in a loop. The models have no access to coordinates, map-data, or external memory scratches - they must navigate using only immediate local neighbourhood observations.<p>There is a model dropdown and you can see how the model has progressed over each phase, constantly getting smaller and increasing its solve rate. Total trained models number in the thousands - I just expose the winning models from each phase.<p>Overall a fun experiment, with much implementation help from AI agents to scaffold and implement the code (I'm a lazy software dev).
Show HN: Infinite Jigsaw Game
Had the idea forever, finally got it working.
Show HN: Optimize and serve models with Fable quality at half the cost
Hi HN, we built world-model-optimizer, an open source tool to continually improve a specialized model for an agent.<p>It does this by simulating production tool responses through text world modeling (similar to QwenAgentWorld, summary here <a href="https://x.com/silennai/status/2073887455884058814" rel="nofollow">https://x.com/silennai/status/2073887455884058814</a>).<p>We can then use this to train a router for frontier, OS, and local models (use defaults or pick which ones to optimize against).<p>wmo ingests agent traces, builds the simulation, embeds the traces, runs different models you choose against the simulation scenarios, and then uses a KNN for model selection (similar to <a href="https://arxiv.org/abs/2505.19797" rel="nofollow">https://arxiv.org/abs/2505.19797</a>).<p>- Cache aware: cache is taken into account for the effective price in routing.<p>- Confidence gated: we don't deviate from the best fit model when paired evidence over retrieved neighbors is below 0.5 standard errors or on queries unlike anything in the fit set.<p>- Optimize for cost or quality: train a balanced, cost max, or quality max router.<p>Usage<p>`wmo build` creates the simulation (or add your own benchmark)<p>`wmo optimize` tunes the router<p>`wmo serve` starts the server and can run everything fully locally. The simulation and router can update over time as more agent traces are gathered and new models are added.<p>Router results vs Fable<p>- RouterBench: -66.5% cost, -1.7% performance, -24.7% latency p50. 77.5% of traffic to Sonnet 5, 16.1% Fable 5.<p>- TauBench: -44.5% cost, +6.3% performance, -20% latency. 83% to Opus 5, 17% to Kimi-K2.6 (over K3).<p>- Terminal Bench 2: -64% cost, +8% performance, -50.6% latency. Sonnet 5 is fully along the pareto front. Training a specialized router per task isn't cheap. In sparse data regimes the value can be "here's the best model".<p>We're working on sample effiient continual learning for agent specific models at experientiallabs.ai"
Show HN: Optimize and serve models with Fable quality at half the cost
Hi HN, we built world-model-optimizer, an open source tool to continually improve a specialized model for an agent.<p>It does this by simulating production tool responses through text world modeling (similar to QwenAgentWorld, summary here <a href="https://x.com/silennai/status/2073887455884058814" rel="nofollow">https://x.com/silennai/status/2073887455884058814</a>).<p>We can then use this to train a router for frontier, OS, and local models (use defaults or pick which ones to optimize against).<p>wmo ingests agent traces, builds the simulation, embeds the traces, runs different models you choose against the simulation scenarios, and then uses a KNN for model selection (similar to <a href="https://arxiv.org/abs/2505.19797" rel="nofollow">https://arxiv.org/abs/2505.19797</a>).<p>- Cache aware: cache is taken into account for the effective price in routing.<p>- Confidence gated: we don't deviate from the best fit model when paired evidence over retrieved neighbors is below 0.5 standard errors or on queries unlike anything in the fit set.<p>- Optimize for cost or quality: train a balanced, cost max, or quality max router.<p>Usage<p>`wmo build` creates the simulation (or add your own benchmark)<p>`wmo optimize` tunes the router<p>`wmo serve` starts the server and can run everything fully locally. The simulation and router can update over time as more agent traces are gathered and new models are added.<p>Router results vs Fable<p>- RouterBench: -66.5% cost, -1.7% performance, -24.7% latency p50. 77.5% of traffic to Sonnet 5, 16.1% Fable 5.<p>- TauBench: -44.5% cost, +6.3% performance, -20% latency. 83% to Opus 5, 17% to Kimi-K2.6 (over K3).<p>- Terminal Bench 2: -64% cost, +8% performance, -50.6% latency. Sonnet 5 is fully along the pareto front. Training a specialized router per task isn't cheap. In sparse data regimes the value can be "here's the best model".<p>We're working on sample effiient continual learning for agent specific models at experientiallabs.ai"
Show HN: Optimize and serve models with Fable quality at half the cost
Hi HN, we built world-model-optimizer, an open source tool to continually improve a specialized model for an agent.<p>It does this by simulating production tool responses through text world modeling (similar to QwenAgentWorld, summary here <a href="https://x.com/silennai/status/2073887455884058814" rel="nofollow">https://x.com/silennai/status/2073887455884058814</a>).<p>We can then use this to train a router for frontier, OS, and local models (use defaults or pick which ones to optimize against).<p>wmo ingests agent traces, builds the simulation, embeds the traces, runs different models you choose against the simulation scenarios, and then uses a KNN for model selection (similar to <a href="https://arxiv.org/abs/2505.19797" rel="nofollow">https://arxiv.org/abs/2505.19797</a>).<p>- Cache aware: cache is taken into account for the effective price in routing.<p>- Confidence gated: we don't deviate from the best fit model when paired evidence over retrieved neighbors is below 0.5 standard errors or on queries unlike anything in the fit set.<p>- Optimize for cost or quality: train a balanced, cost max, or quality max router.<p>Usage<p>`wmo build` creates the simulation (or add your own benchmark)<p>`wmo optimize` tunes the router<p>`wmo serve` starts the server and can run everything fully locally. The simulation and router can update over time as more agent traces are gathered and new models are added.<p>Router results vs Fable<p>- RouterBench: -66.5% cost, -1.7% performance, -24.7% latency p50. 77.5% of traffic to Sonnet 5, 16.1% Fable 5.<p>- TauBench: -44.5% cost, +6.3% performance, -20% latency. 83% to Opus 5, 17% to Kimi-K2.6 (over K3).<p>- Terminal Bench 2: -64% cost, +8% performance, -50.6% latency. Sonnet 5 is fully along the pareto front. Training a specialized router per task isn't cheap. In sparse data regimes the value can be "here's the best model".<p>We're working on sample effiient continual learning for agent specific models at experientiallabs.ai"