Here’s how to build your own portable offline AI on your Mac OS. You will need to have an LLM file and a GGUF AI model like the Qwen model. First, you will need to download the latest LLM file from the GitHub Releases page under Assets. Then, download the AI model you want to use. For this example, the Qwen model is being used, but you could try out other compatible models as well. After that, keep them inside the same folder.

Then, download the Launcher.sh file from the link below and edit it in TextEdit. You will only need to change the exact LLM file name and the exact model file name with the .gguf extension. After that, just double-click the Launcher to start up the offline AI. It could take a while to load depending on your Mac and the model you’re using. Once it starts up, you will get a local link. Copy that link and paste it into any browser.

Now you will have your Offline AI LLM running on your macOS. You can ask any question you want, and this works completely offline without an internet connection.

Thank you very much.