john-rocky/coreai-model-zoo

CoreAI-Model-Zoo is a community collection of AI language models that have been converted to run natively on Apple devices using Apple's Core AI framework, which is part of iOS 27 and macOS 27. Core AI is Apple's successor to their earlier Core ML format, and models in this format run locally on the device without sending data to an external server. The repository hosts converted versions of models from several families, including Qwen3.5, Gemma 4, GLM-4.7, Granite, and LFM2.5, in sizes ranging from under 1 billion parameters to 35 billion. Each model is available for download from Hugging Face as an .aimodel bundle. Smaller models like Qwen3.5-0.8B run on iPhone, while the larger ones require a Mac with Apple Silicon. The README includes measured decode speeds in tokens per second for each model on an iPhone 17 Pro (both GPU and Apple Neural Engine) and on an M4 Max Mac. Some of the included models required custom Metal shaders to run at all or to run at practical speeds. The Gemma 4 12B and 31B models crash with Apple's stock attention implementation on long sequences due to a memory overflow bug in the GPU scheduler, so the project ships a custom flash-decode kernel for those layers. The MoE (mixture-of-experts) models received a custom kernel that reads only the active expert weights rather than the full weight matrix, which gave a 3.6x speed improvement for one of the models. A companion iPhone and Mac app called CoreAIChat is included that lets you download and run the models without building any code. It requires an iOS 27 or macOS 27 beta device, since the .aimodel bundle format is only available in those beta SDKs. The repository also includes a knowledge base covering conversion steps, common pitfalls, how compute unit selection affects speed, and the custom kernel implementations. Vision-language models that handle both images and text are included alongside text-only models and an embedding model suited for on-device search.