The best Hacker News stories from Show from the past day
Latest posts:
Show HN: There are over a thousand possible finger arrangements for your hands
Show HN: There are over a thousand possible finger arrangements for your hands
Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching
Hello hacker news,<p>I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: <a href="https://github.com/BerriAI/litellm/">https://github.com/BerriAI/litellm/</a><p>We’re open sourcing our implementation of liteLLM proxy: <a href="https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-server/readme.md">https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-...</a><p>TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming<p>What can liteLLM proxy do?
- It’s a central place to manage all LLM provider integrations<p>- Consistent Input/Output Format
- Call all models using the OpenAI format: completion(model, messages)
- Text responses will always be available at ['choices'][0]['message']['content']<p>- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)<p>- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone<p>- Token Usage & Spend - Track Input + Completion tokens used + Spend/model<p>- Caching - Implementation of Semantic Caching<p>- Streaming & Async Support - Return generators to stream text responses<p>You can deploy liteLLM to your own infrastructure using Railway, GCP, AWS, Azure<p>Happy completion() !
Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching
Hello hacker news,<p>I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: <a href="https://github.com/BerriAI/litellm/">https://github.com/BerriAI/litellm/</a><p>We’re open sourcing our implementation of liteLLM proxy: <a href="https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-server/readme.md">https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-...</a><p>TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming<p>What can liteLLM proxy do?
- It’s a central place to manage all LLM provider integrations<p>- Consistent Input/Output Format
- Call all models using the OpenAI format: completion(model, messages)
- Text responses will always be available at ['choices'][0]['message']['content']<p>- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)<p>- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone<p>- Token Usage & Spend - Track Input + Completion tokens used + Spend/model<p>- Caching - Implementation of Semantic Caching<p>- Streaming & Async Support - Return generators to stream text responses<p>You can deploy liteLLM to your own infrastructure using Railway, GCP, AWS, Azure<p>Happy completion() !
Show HN: liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching
Hello hacker news,<p>I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: <a href="https://github.com/BerriAI/litellm/">https://github.com/BerriAI/litellm/</a><p>We’re open sourcing our implementation of liteLLM proxy: <a href="https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-server/readme.md">https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-...</a><p>TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming<p>What can liteLLM proxy do?
- It’s a central place to manage all LLM provider integrations<p>- Consistent Input/Output Format
- Call all models using the OpenAI format: completion(model, messages)
- Text responses will always be available at ['choices'][0]['message']['content']<p>- Error Handling Using Model Fallbacks (if GPT-4 fails, try llama2)<p>- Logging - Log Requests, Responses and Errors to Supabase, Posthog, Mixpanel, Sentry, Helicone<p>- Token Usage & Spend - Track Input + Completion tokens used + Spend/model<p>- Caching - Implementation of Semantic Caching<p>- Streaming & Async Support - Return generators to stream text responses<p>You can deploy liteLLM to your own infrastructure using Railway, GCP, AWS, Azure<p>Happy completion() !
Show HN: Covert – Rewrite of HashiCorp Vault Using Rust, SQLite and Litestream
Show HN: Covert – Rewrite of HashiCorp Vault Using Rust, SQLite and Litestream
Show HN: Pykoi – a Python library for LLM data collection and fine tuning
Hi HN,<p>pykoi is an open-source python library for ML scientists. pykoi makes it easier to collect data for LLMs, to use that data for finetuning, and to compare models to each other (e.g. your model pre- and post- finetuning, or your model vs openai vs claude). The library comes from pain points we experienced in LLM development:<p>1. Collecting feedback data from users isn't as easy as it could be. (The current process usually involves sharing excel files of annotated responses back-and-forth, offering no insight into how users actually engage with your models).<p>2. RLHF remains complicated to carry out. By <i>complicated</i>, we mean requires a lot of steps, hundreds of configs, lengthy setups, etc.<p>3. Comparing models to each other <i>as they're used</i> (that is, independent from academic metrics) is full of friction. The current approach: spin up a model, ask questions, write them down. Repeat for other models then compare.<p>At a high-level, we think that the active learning process should be closed-loop: data collection, fine tuning, and inference all feed from the same system. This library is our first step in that direction.<p>The project is still very early but we hope that some if it is useful. Note, we're fully open-source, and actively adding features!<p>Website: <a href="https://www.cambioml.com/pykoi">https://www.cambioml.com/pykoi</a>
GitHub: <a href="https://github.com/CambioML/pykoi">https://github.com/CambioML/pykoi</a><p>We would love your feedback!
Show HN: Pykoi – a Python library for LLM data collection and fine tuning
Hi HN,<p>pykoi is an open-source python library for ML scientists. pykoi makes it easier to collect data for LLMs, to use that data for finetuning, and to compare models to each other (e.g. your model pre- and post- finetuning, or your model vs openai vs claude). The library comes from pain points we experienced in LLM development:<p>1. Collecting feedback data from users isn't as easy as it could be. (The current process usually involves sharing excel files of annotated responses back-and-forth, offering no insight into how users actually engage with your models).<p>2. RLHF remains complicated to carry out. By <i>complicated</i>, we mean requires a lot of steps, hundreds of configs, lengthy setups, etc.<p>3. Comparing models to each other <i>as they're used</i> (that is, independent from academic metrics) is full of friction. The current approach: spin up a model, ask questions, write them down. Repeat for other models then compare.<p>At a high-level, we think that the active learning process should be closed-loop: data collection, fine tuning, and inference all feed from the same system. This library is our first step in that direction.<p>The project is still very early but we hope that some if it is useful. Note, we're fully open-source, and actively adding features!<p>Website: <a href="https://www.cambioml.com/pykoi">https://www.cambioml.com/pykoi</a>
GitHub: <a href="https://github.com/CambioML/pykoi">https://github.com/CambioML/pykoi</a><p>We would love your feedback!
Show HN: Pykoi – a Python library for LLM data collection and fine tuning
Hi HN,<p>pykoi is an open-source python library for ML scientists. pykoi makes it easier to collect data for LLMs, to use that data for finetuning, and to compare models to each other (e.g. your model pre- and post- finetuning, or your model vs openai vs claude). The library comes from pain points we experienced in LLM development:<p>1. Collecting feedback data from users isn't as easy as it could be. (The current process usually involves sharing excel files of annotated responses back-and-forth, offering no insight into how users actually engage with your models).<p>2. RLHF remains complicated to carry out. By <i>complicated</i>, we mean requires a lot of steps, hundreds of configs, lengthy setups, etc.<p>3. Comparing models to each other <i>as they're used</i> (that is, independent from academic metrics) is full of friction. The current approach: spin up a model, ask questions, write them down. Repeat for other models then compare.<p>At a high-level, we think that the active learning process should be closed-loop: data collection, fine tuning, and inference all feed from the same system. This library is our first step in that direction.<p>The project is still very early but we hope that some if it is useful. Note, we're fully open-source, and actively adding features!<p>Website: <a href="https://www.cambioml.com/pykoi">https://www.cambioml.com/pykoi</a>
GitHub: <a href="https://github.com/CambioML/pykoi">https://github.com/CambioML/pykoi</a><p>We would love your feedback!
Show HN: Pykoi – a Python library for LLM data collection and fine tuning
Hi HN,<p>pykoi is an open-source python library for ML scientists. pykoi makes it easier to collect data for LLMs, to use that data for finetuning, and to compare models to each other (e.g. your model pre- and post- finetuning, or your model vs openai vs claude). The library comes from pain points we experienced in LLM development:<p>1. Collecting feedback data from users isn't as easy as it could be. (The current process usually involves sharing excel files of annotated responses back-and-forth, offering no insight into how users actually engage with your models).<p>2. RLHF remains complicated to carry out. By <i>complicated</i>, we mean requires a lot of steps, hundreds of configs, lengthy setups, etc.<p>3. Comparing models to each other <i>as they're used</i> (that is, independent from academic metrics) is full of friction. The current approach: spin up a model, ask questions, write them down. Repeat for other models then compare.<p>At a high-level, we think that the active learning process should be closed-loop: data collection, fine tuning, and inference all feed from the same system. This library is our first step in that direction.<p>The project is still very early but we hope that some if it is useful. Note, we're fully open-source, and actively adding features!<p>Website: <a href="https://www.cambioml.com/pykoi">https://www.cambioml.com/pykoi</a>
GitHub: <a href="https://github.com/CambioML/pykoi">https://github.com/CambioML/pykoi</a><p>We would love your feedback!
Show HN: Obl.ong, Free, quality domains for all
Show HN: Obl.ong, Free, quality domains for all
Show HN: Pip Imports in Deno
deno_python 0.3.1 adds support for importing python pip packages directly in JavaScript! Fun and useful, slightly cursed.
Show HN: Pip Imports in Deno
deno_python 0.3.1 adds support for importing python pip packages directly in JavaScript! Fun and useful, slightly cursed.
Show HN: Pip Imports in Deno
deno_python 0.3.1 adds support for importing python pip packages directly in JavaScript! Fun and useful, slightly cursed.
Show HN: Pip Imports in Deno
deno_python 0.3.1 adds support for importing python pip packages directly in JavaScript! Fun and useful, slightly cursed.
Show HN: Tetris, but the blocks are ARM instructions that execute in the browser
OFRAK Tetris is a project I started at work about two weeks ago. It's a web-based game that works on desktop and mobile. I made it for my company to bring to events like DEF CON, and to promote our binary analysis and patching framework called OFRAK.<p>In the game, 32-bit, little-endian ARM assembly instructions fall, and you can modify the operands before executing them on a CPU emulator. There are two segments mapped – one for instructions, and one for data (though both have read, write, and execute permissions). Your score is a four byte signed integer stored at the virtual address pointed to by the R12 register, and the goal is to use the instructions that fall to make the score value in memory as high as possible. When it's game over, you can download your game as an ELF to relive the glory in GDB on your favorite ARM device.<p>The CPU emulator is a version of Unicorn (<a href="https://www.unicorn-engine.org/" rel="nofollow noreferrer">https://www.unicorn-engine.org/</a>) that has been cross-compiled to WebAssembly (<a href="https://alexaltea.github.io/unicorn.js/" rel="nofollow noreferrer">https://alexaltea.github.io/unicorn.js/</a>), so everything on the page runs in the browser without the need for any complicated infrastructure on the back end.<p>Since I've only been working on this for a short period of time leading up to its debut at DEF CON, there are still many more features I'd eventually like to implement. These include adding support for other ISAs besides ARM, adding an instruction reference manual, and lots of little cleanups, bug fixes, and adjustments.<p>My highest score is 509,644,979, but my average is about 131,378.<p>I look forward to feedback, bug reports, feature requests, and strategy discussions!
Show HN: Tetris, but the blocks are ARM instructions that execute in the browser
OFRAK Tetris is a project I started at work about two weeks ago. It's a web-based game that works on desktop and mobile. I made it for my company to bring to events like DEF CON, and to promote our binary analysis and patching framework called OFRAK.<p>In the game, 32-bit, little-endian ARM assembly instructions fall, and you can modify the operands before executing them on a CPU emulator. There are two segments mapped – one for instructions, and one for data (though both have read, write, and execute permissions). Your score is a four byte signed integer stored at the virtual address pointed to by the R12 register, and the goal is to use the instructions that fall to make the score value in memory as high as possible. When it's game over, you can download your game as an ELF to relive the glory in GDB on your favorite ARM device.<p>The CPU emulator is a version of Unicorn (<a href="https://www.unicorn-engine.org/" rel="nofollow noreferrer">https://www.unicorn-engine.org/</a>) that has been cross-compiled to WebAssembly (<a href="https://alexaltea.github.io/unicorn.js/" rel="nofollow noreferrer">https://alexaltea.github.io/unicorn.js/</a>), so everything on the page runs in the browser without the need for any complicated infrastructure on the back end.<p>Since I've only been working on this for a short period of time leading up to its debut at DEF CON, there are still many more features I'd eventually like to implement. These include adding support for other ISAs besides ARM, adding an instruction reference manual, and lots of little cleanups, bug fixes, and adjustments.<p>My highest score is 509,644,979, but my average is about 131,378.<p>I look forward to feedback, bug reports, feature requests, and strategy discussions!
Show HN: Tetris, but the blocks are ARM instructions that execute in the browser
OFRAK Tetris is a project I started at work about two weeks ago. It's a web-based game that works on desktop and mobile. I made it for my company to bring to events like DEF CON, and to promote our binary analysis and patching framework called OFRAK.<p>In the game, 32-bit, little-endian ARM assembly instructions fall, and you can modify the operands before executing them on a CPU emulator. There are two segments mapped – one for instructions, and one for data (though both have read, write, and execute permissions). Your score is a four byte signed integer stored at the virtual address pointed to by the R12 register, and the goal is to use the instructions that fall to make the score value in memory as high as possible. When it's game over, you can download your game as an ELF to relive the glory in GDB on your favorite ARM device.<p>The CPU emulator is a version of Unicorn (<a href="https://www.unicorn-engine.org/" rel="nofollow noreferrer">https://www.unicorn-engine.org/</a>) that has been cross-compiled to WebAssembly (<a href="https://alexaltea.github.io/unicorn.js/" rel="nofollow noreferrer">https://alexaltea.github.io/unicorn.js/</a>), so everything on the page runs in the browser without the need for any complicated infrastructure on the back end.<p>Since I've only been working on this for a short period of time leading up to its debut at DEF CON, there are still many more features I'd eventually like to implement. These include adding support for other ISAs besides ARM, adding an instruction reference manual, and lots of little cleanups, bug fixes, and adjustments.<p>My highest score is 509,644,979, but my average is about 131,378.<p>I look forward to feedback, bug reports, feature requests, and strategy discussions!