Plenty of people are excited about running open source LLMs on their own computer. You don’t need to have a giant PC to get started. The Tiiny AI Pocket Lab is a nifty portable computer that lets you run up to 120B models. It is compatible with OpenAI API, so you will be able to use it with LangChain, AutoGPT, and other workflows. Here are the main specs:
More gadgets like this: here
- ARMv9.2 processor with NPU, 30 INT8 TOPS
- dNPU with 160 INT8 TOPS performance
- 80GB RAM
- 1TB SSD
- Bluetooth 5.3 & WiFi
- 3 x USB-C
Tiiny AI First Look & Testing - A Portable Local AI Powerhouse!
This computer has a noise rating of under 35dB. The entire unit weighs 300g, so you can easily carry it in your pocket. It is compatible with Windows and macOS devices. Simply plug it in, select your model and agent, and the rest is easy.
**Disclaimer: Our posts may feature aff links. As an Amazon Associate, we earn from qualifying purchases. Please read our disclaimer on how we fund this site.
