Skip to content

Local AI tooling

Local AI

A practical starting point for running open-source models on Apple Silicon.

View on GitHubOpen source · MIT

Run something first. Learn the rest as you go.

Local models come with a lot of choices: models, runtimes, formats, quantizations, context sizes, and configuration. You don't need to understand all of that before trying one.

I keep the runtime guides, conceptual notes, and model examples together so the first attempt stays practical. The point is to have somewhere useful to go when the next choice gets confusing.

Start with the part you need.

Learn the basics

New to local AI? Start with plain-language explanations of models, runtimes, quantization, GGUF, context, memory, and the terminology you'll encounter.

Read the docs →

Pick a starting model

See models I've actually used, including the artifact, runtime, and machine where they gave me a useful experience.

Browse local models →

Start here

Install it where you work.

Clone the repository to keep the commands and implementation details close at hand. The project focuses on Apple Silicon and stays a practical reference rather than a packaged inference runtime.

git clone https://github.com/joacod/local-ai.git
cd local-ai

Use it

Pick a runtime. Get a model. Run it.

Choose the path that matches your starting point:

What to expect

  • Start with a working path; learn the terminology when it becomes relevant.
  • The examples are practical starting points, not universal recommendations.
  • GitHub holds the commands and implementation details.

A starting point, not a leaderboard.

Local AI isn't a benchmark suite or an exhaustive model catalog. The model and machine notes document combinations I've actually used so they can give someone with similar hardware a useful place to start.