The progress of open source is too fast!
Just one file, easily deploy LLama on personal computers!
Source code
Distribute and run LLMs with a single file.
llamafile is an open source project, its main feature is to allow developers and end users to distribute and run large language models (LLM) using a single file. Here is a detailed introduction to the llamafile project:
Project goal: The goal of the llamafile project is to simplify access to and use of large language models. With this project, users can easily run LLM without complex installation and configuration processes.
Technical implementation: To achieve this goal, llamafile combines llama.cpp with Cosmopolitan Libc into one framework. This combination compresses all the complexity of LLM into a single executable file, which can be run natively on multiple operating systems, including macOS, Windows, Linux, FreeBSD, OpenBSD, and NetBSD.
Usability: Users only need to download the corresponding llamafile and perform simple operations based on different operating systems (such as adding the .exe extension and double-clicking to run on Windows) to start LLM. In addition, llamafile provides a WebUI interface, which allows users to interact with LLM more conveniently.
Supported models: Currently, llamafile supports various large language models, including LLaVA, Mistral, Mixtral, and WizardCoder, etc. These models are all quantized models, so they can run smoothly even in pure CPU environments.
Community support: The llamafile project is hosted on GitHub and has received considerable attention. In just two months, the number of Stars for this project has exceeded 10,000, indicating the recognition and interest of developers and users in this project.
In summary, llamafile is an open source project aimed at simplifying the distribution and operation of large language models. By compressing a complex LLM into a single executable file, it greatly reduces the threshold for user use, allowing more people to easily experience and leverage the powerful functions of large language models.
The easiest way to try it yourself is to download the example llamafile of the LLaVA model (License: LLaMA 2, OpenAI). LLaVA is a new LLM that can do more than just chat. You can also upload images and ask questions about them. For llamafile, all of this happens locally; no data leaves your computer.
Download llava-v1.5-7b-q4.llamafile (3.97 GB).
Download lava-v1.5-7b-q4.llama file (3.97 GB).
Open your computer's terminal.
Open the terminal of your computer.
If you're using macOS, Linux, or BSD, you'll need to grant permission for your computer to execute this new file. (You only need to do this once.)
If you're using macOS, Linux, or BSD, you'll need to grant permission for your computer to execute this new file. (You only need to do this once.)
chmod +x llava-v1.5-7b-q4.llamafile
chmod +x llava-v1.5-7b-q4.llamafile
If you're on Windows, rename the file by adding ".exe" on the end.
If you're on Windows, rename the file by adding ".exe" at the end.
Run the llamafile. e.g.:
Run the llamafile. e.g.:
./llava-v1.5-7b-q4.llamafile
./llava-v1.5-7b-q4.llamafile
Your browser should open automatically and display a chat interface. (If it doesn't, just open your browser and point it at http://localhost:8080)
Your browser should open automatically and display a chat interface. (If it doesn't, just open your browser and point it at http://localhost:8080)
When you're done chatting, return to your terminal and hit Control-C to shut down llamafile.
When you're done chatting, return to your terminal and hit Control-C to shut down llamafile.