The best Hacker News stories from Show from the past day
Latest posts:
Show HN: Crunching 1,200 Authors' Favorite Reads of 2023
Hi all, creator here :)<p>I launched Shepherd.com on a Show HN in April 2021 (<a href="https://news.ycombinator.com/item?id=26871660">https://news.ycombinator.com/item?id=26871660</a>), and it has come a long way.<p>My goal with Shepherd is to create an experience that feels like wandering your local bookstore, along with little notes from authors & experts sharing why each book is one of their all-time favorites.<p>For 2023, I surveyed 1,200+ authors to get their three favorite reads of the year. Then I crunched that data and broke it by genre, age range, and when it was published. Publisher data is a nightmare, so some of the genres are not perfect; I am working on improving that and some of the NLP/ML that drives this.<p>Check out their top sci-fi reads: <a href="https://shepherd.com/bboy/2023/science-fiction" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/science-fiction</a><p>Or, their top nonfiction reads: <a href="https://shepherd.com/bboy/2023/nonfiction" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/nonfiction</a><p>You can also zoom in on each author’s favorite 3 reads.<p>Louise Carey - <a href="https://shepherd.com/bboy/2023/f/louise-carey" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/f/louise-carey</a><p>Kevin Klehr - <a href="https://shepherd.com/bboy/2023/f/kevin-klehr" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/f/kevin-klehr</a><p>Alice C. Hill - <a href="https://shepherd.com/bboy/2023/f/alice-c-hill" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/f/alice-c-hill</a><p>Sara Ackerman - <a href="https://shepherd.com/bboy/2023/f/sara-ackerman" rel="nofollow noreferrer">https://shepherd.com/bboy/2023/f/sara-ackerman</a><p>My email is ben@shepherd.com if you want to share ideas or suggestions for 2024.<p>Thanks, Ben<p>P.S. I have a newsletter for readers here where I share what I am building, new features, my fav book lists: <a href="https://forauthors.shepherd.com/newsletter-for-readers" rel="nofollow noreferrer">https://forauthors.shepherd.com/newsletter-for-readers</a>
Show HN: I built an OSS alternative to Azure OpenAI services
Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services.<p>Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure.<p>"How can I track LLM spend per API key?"<p>"Can I create a development OpenAI API key with limited access for Bob?"<p>"Can I see my LLM spend breakdown by models and endpoints?"<p>"Can I create 100 OpenAI API keys that my students could use in a classroom setting?"<p>These are questions that BricksLLM helps you answer.<p>BricksLLM is an API gateway that let you create API keys with rate limit, cost control and ttl that could be used to access all OpenAI and Anthropic endpoints with out of box analytics.<p>When I first started building with OpenAI APIs, I was constantly worried about API keys being comprised since vanilla OpenAI API keys would grant you unlimited access to all of their models. There are stories of people losing thousands of dollars and the existence of a black market for stolen OpenAI API keys.<p>This is why I started building a proxy for ourselves that allows for the creation of API keys with rate limits and cost controls. I built BricksLLM in Go since that was the language I used to build performative ads exchanges that scaled to thousands of requests per second at my previous job. A lot of developer tools in LLM ops are built with Python which I believe might be suboptimal in terms of performance and compute resource efficiency.<p>One of the challenges building this platform is to get accurate token counts for different OpenAI and Anthropic models. LLM providers are not exactly transparent with the way how they count prompt and completion tokens. In addition to user input, OpenAI and Anthropic pad prompt inputs with additional instructions or phrases that contribute to the final token counts. For example, Anthropic's actual completion token consumption is consistently 4 more than the token count of the completion output.<p>The latency of the gateway hovers around 50ms. Half of the latency comes from the tokenizer. If I start utilizing Go routines, might be able to lower the latency of the gateway to 30ms.<p>BricksLLM is not an observability platform, but we do provide integration with Datadog so you can get more insights regarding what is going on inside the proxy. Compared to other tools in the LLMOps space, I believe that BricksLLM has the most comprehensive features when it comes to access control.<p>Let me know what you guys think.
Show HN: I built an OSS alternative to Azure OpenAI services
Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services.<p>Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure.<p>"How can I track LLM spend per API key?"<p>"Can I create a development OpenAI API key with limited access for Bob?"<p>"Can I see my LLM spend breakdown by models and endpoints?"<p>"Can I create 100 OpenAI API keys that my students could use in a classroom setting?"<p>These are questions that BricksLLM helps you answer.<p>BricksLLM is an API gateway that let you create API keys with rate limit, cost control and ttl that could be used to access all OpenAI and Anthropic endpoints with out of box analytics.<p>When I first started building with OpenAI APIs, I was constantly worried about API keys being comprised since vanilla OpenAI API keys would grant you unlimited access to all of their models. There are stories of people losing thousands of dollars and the existence of a black market for stolen OpenAI API keys.<p>This is why I started building a proxy for ourselves that allows for the creation of API keys with rate limits and cost controls. I built BricksLLM in Go since that was the language I used to build performative ads exchanges that scaled to thousands of requests per second at my previous job. A lot of developer tools in LLM ops are built with Python which I believe might be suboptimal in terms of performance and compute resource efficiency.<p>One of the challenges building this platform is to get accurate token counts for different OpenAI and Anthropic models. LLM providers are not exactly transparent with the way how they count prompt and completion tokens. In addition to user input, OpenAI and Anthropic pad prompt inputs with additional instructions or phrases that contribute to the final token counts. For example, Anthropic's actual completion token consumption is consistently 4 more than the token count of the completion output.<p>The latency of the gateway hovers around 50ms. Half of the latency comes from the tokenizer. If I start utilizing Go routines, might be able to lower the latency of the gateway to 30ms.<p>BricksLLM is not an observability platform, but we do provide integration with Datadog so you can get more insights regarding what is going on inside the proxy. Compared to other tools in the LLMOps space, I believe that BricksLLM has the most comprehensive features when it comes to access control.<p>Let me know what you guys think.
Show HN: I built an OSS alternative to Azure OpenAI services
Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services.<p>Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure.<p>"How can I track LLM spend per API key?"<p>"Can I create a development OpenAI API key with limited access for Bob?"<p>"Can I see my LLM spend breakdown by models and endpoints?"<p>"Can I create 100 OpenAI API keys that my students could use in a classroom setting?"<p>These are questions that BricksLLM helps you answer.<p>BricksLLM is an API gateway that let you create API keys with rate limit, cost control and ttl that could be used to access all OpenAI and Anthropic endpoints with out of box analytics.<p>When I first started building with OpenAI APIs, I was constantly worried about API keys being comprised since vanilla OpenAI API keys would grant you unlimited access to all of their models. There are stories of people losing thousands of dollars and the existence of a black market for stolen OpenAI API keys.<p>This is why I started building a proxy for ourselves that allows for the creation of API keys with rate limits and cost controls. I built BricksLLM in Go since that was the language I used to build performative ads exchanges that scaled to thousands of requests per second at my previous job. A lot of developer tools in LLM ops are built with Python which I believe might be suboptimal in terms of performance and compute resource efficiency.<p>One of the challenges building this platform is to get accurate token counts for different OpenAI and Anthropic models. LLM providers are not exactly transparent with the way how they count prompt and completion tokens. In addition to user input, OpenAI and Anthropic pad prompt inputs with additional instructions or phrases that contribute to the final token counts. For example, Anthropic's actual completion token consumption is consistently 4 more than the token count of the completion output.<p>The latency of the gateway hovers around 50ms. Half of the latency comes from the tokenizer. If I start utilizing Go routines, might be able to lower the latency of the gateway to 30ms.<p>BricksLLM is not an observability platform, but we do provide integration with Datadog so you can get more insights regarding what is going on inside the proxy. Compared to other tools in the LLMOps space, I believe that BricksLLM has the most comprehensive features when it comes to access control.<p>Let me know what you guys think.
Show HN: I Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real
Show HN: I created an extension for direct streaming of videos, bypassing ads
Show HN: Rot - Offline secrets management
Finally open sourced an internal tool we've been using for managing secrets. It's similar to SOPS, but more opinionated, easier to configure/use correctly, and produces nicer git diffs. It also supports one-way encryption, so you don't have to know the private key to add secrets.
Show HN: Rot - Offline secrets management
Finally open sourced an internal tool we've been using for managing secrets. It's similar to SOPS, but more opinionated, easier to configure/use correctly, and produces nicer git diffs. It also supports one-way encryption, so you don't have to know the private key to add secrets.
Show HN: Rot - Offline secrets management
Finally open sourced an internal tool we've been using for managing secrets. It's similar to SOPS, but more opinionated, easier to configure/use correctly, and produces nicer git diffs. It also supports one-way encryption, so you don't have to know the private key to add secrets.
Show HN: Watering my Christmas tree with ESPHome
Show HN: Watering my Christmas tree with ESPHome
Show HN: Watering my Christmas tree with ESPHome
Show HN: Open source alternative to ChatGPT and ChatPDF-like AI tools
Hey everyone,<p>We have been building SecureAI Tools -- an open-source application layer for ChatGPT and ChatPDF-like AI tools.<p>It works with locally running LLMs as well as with OpenAI-compatible APIs. For local LLMs, it supports Ollama which supports all the gguf/ggml models.<p>Currently, it has two features: Chat-with-LLM, and Chat-with-PDFs. It is optimized for self-hosting use cases and comes with basic user management features.<p>Here are some quick demos:<p><pre><code> * Chat with documents using OpenAI's GPT3.5 model: https://www.youtube.com/watch?v=Br2D3G9O47s
* Chat with documents using a locally running Mistral model (M2 MacBook): https://www.youtube.com/watch?v=UvRHL6f_w74
</code></pre>
Hope you all like it :)
Show HN: We've built an Open Source plugin to make using AI easier for VFX
Hi, I’m Bryan Lyon, the CTO of DeepMake. We’re making plugins for VFX software suites (Right now we have a pre-release of our After Effects plugin ready) that enable you to use Open Source AI tools without leaving the workflow. It makes all the AI tools work just like another plugin in the VFX software.<p>In the past, I worked on VFX adjacent AI and was more than once asked to make AI tools easier to use so the VFX experts could use them. In many cases my job was little more than making it easy to install and make sense to use. After writing dozens of “one-off” scripts that made an AI tool usable I decided there should be a better way. To that end we’ve simplified the process. Right now our public release is still not the easiest to install, but once you’ve installed one plugin and the backend, installing new plugins is as simple as extracting a zip file or cloning a repo (and we’re going to make it even easier).<p>Our target audience isn’t non-technical people, but rather people who don’t have the time or experience to go through installing an AI tool which may not even include an easy way to send your own data through the pipeline. We hope to make VFX easier for people who can’t really do VFX now due to either a lack of time or experience. We also want to make VFX more powerful for VFX experts.<p>We’ve built the backend to make plugins extremely easy to make and hope that the community will be able to take advantage of an easy way to integrate with VFX tools. IN the spirit of open source we’ve released our code as open source (We’re cleaning up the AE plugin side, but it’ll be released too).<p>We plan on monetizing based on managing massive plugins across compute farms for big VFX shops and other features that benefit those larger teams, the core of the system will remain open source and freely available.<p>So far, we’ve released a Stable Diffusion plugin, and have more including Masking, Super Resolution, and more coming up in the pipeline.<p>I hope you like our software and look forward to hearing from you.
Show HN: We've built an Open Source plugin to make using AI easier for VFX
Hi, I’m Bryan Lyon, the CTO of DeepMake. We’re making plugins for VFX software suites (Right now we have a pre-release of our After Effects plugin ready) that enable you to use Open Source AI tools without leaving the workflow. It makes all the AI tools work just like another plugin in the VFX software.<p>In the past, I worked on VFX adjacent AI and was more than once asked to make AI tools easier to use so the VFX experts could use them. In many cases my job was little more than making it easy to install and make sense to use. After writing dozens of “one-off” scripts that made an AI tool usable I decided there should be a better way. To that end we’ve simplified the process. Right now our public release is still not the easiest to install, but once you’ve installed one plugin and the backend, installing new plugins is as simple as extracting a zip file or cloning a repo (and we’re going to make it even easier).<p>Our target audience isn’t non-technical people, but rather people who don’t have the time or experience to go through installing an AI tool which may not even include an easy way to send your own data through the pipeline. We hope to make VFX easier for people who can’t really do VFX now due to either a lack of time or experience. We also want to make VFX more powerful for VFX experts.<p>We’ve built the backend to make plugins extremely easy to make and hope that the community will be able to take advantage of an easy way to integrate with VFX tools. IN the spirit of open source we’ve released our code as open source (We’re cleaning up the AE plugin side, but it’ll be released too).<p>We plan on monetizing based on managing massive plugins across compute farms for big VFX shops and other features that benefit those larger teams, the core of the system will remain open source and freely available.<p>So far, we’ve released a Stable Diffusion plugin, and have more including Masking, Super Resolution, and more coming up in the pipeline.<p>I hope you like our software and look forward to hearing from you.
Show HN: Fine-grained stylistic control of LLMs using model arithmetic
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: <a href="https://arxiv.org/abs/2311.14479" rel="nofollow noreferrer">https://arxiv.org/abs/2311.14479</a>. Feedback and potential applications are welcome.
Show HN: Fine-grained stylistic control of LLMs using model arithmetic
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: <a href="https://arxiv.org/abs/2311.14479" rel="nofollow noreferrer">https://arxiv.org/abs/2311.14479</a>. Feedback and potential applications are welcome.
Show HN: Fine-grained stylistic control of LLMs using model arithmetic
We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: <a href="https://arxiv.org/abs/2311.14479" rel="nofollow noreferrer">https://arxiv.org/abs/2311.14479</a>. Feedback and potential applications are welcome.
Show HN: Minimalist Calendars Generator
Show HN: Minimalist Calendars Generator