Signals Inbox·August 26, 2026·AI Infrastructure

Can you easily run a local LLM on the new Mac computers?

Yes. On the newest Macs, running a useful local LLM is now straightforward, and 24–32GB of unified memory is enough to make local AI genuinely practical rather than a hobbyist demo. The interesting part starts above that: 64GB and 128GB Macs can run models that ordinary consumer GPUs simply cannot hold.

Q1Why are the newest Macs suddenly so good for local LLMs?

The newest Macs have become much more interesting for local LLMs because Apple is putting unusually large amounts of fast unified memory into computers that still look and behave like normal Macs.

The change has accelerated lately. The current M5 Max MacBook Pro reaches 128GB of unified memory with up to 614GB/s of bandwidth. Apple has now also announced an M5 Ultra Mac Studio that goes all the way to 512GB and 1.2TB/s, while the new M6 Mac mini reaches 32GB and 170GB/s.

Those numbers are especially relevant for LLMs because model weights have to sit somewhere in memory. A consumer Nvidia GPU might have 16GB or 24GB of VRAM even when the PC itself has much more system RAM. A Mac can give its GPU access to a much larger shared pool.

There is one important freshness caveat. The M6 Mac mini and M5 Ultra Mac Studio have only just been announced and have not yet reached ordinary buyers. Apple has published performance tests using preproduction machines, but independent M5 Ultra LLM benchmarks are still missing. So we already know the memory capacity and bandwidth; we do not yet know exactly how fast every model will run on the new top-end Studio.

Q2Is running a local LLM on a Mac actually easy now?

Running a local LLM on an Apple Silicon Mac is currently easy enough that most people can do it without knowing much about machine learning.

LM Studio is probably the clearest example. We can install a normal Mac application, search for a model, download it and start chatting through a graphical interface. Ollama takes a slightly more developer-oriented approach but can launch models with a short command and expose them to other applications through a local API.

LM Studio currently recommends at least 16GB of memory on Mac, although its documentation says 8GB machines can still use smaller models with limited context. Once a model has been downloaded, LM Studio can also work entirely offline.

The underlying software has improved too. Apple's MLX framework was built specifically around Apple Silicon and its shared memory architecture. Tools around MLX now offer ready-made versions of popular models, so getting GPU acceleration no longer requires us to build our own Metal inference stack.

For ordinary experimentation, the installation problem has largely been solved. The part we still have to pay attention to is choosing a model that actually fits the Mac.

Q3Which new Macs are actually good for local LLMs?

Some new Macs are excellent local-LLM machines, while the cheapest models remain quite limited because Apple currently sells Macs with everything from 8GB to 512GB of unified memory.

The MacBook Neo is the obvious warning. It is a new Mac, but Apple gives it just 8GB of unified memory and 60GB/s of bandwidth. That is enough for small quantized models, although it falls below LM Studio's recommended 16GB.

The new M6 Mac mini is much more interesting. It can be configured with up to 32GB and offers as much as 170GB/s of bandwidth. Above that, M5 Pro systems reach 64GB, while M5 Max machines reach 128GB and much higher bandwidth.

At the extreme end, the newly announced M5 Ultra Mac Studio can be configured with 512GB. Apple says the 512GB version will arrive later than the rest of the new Studio lineup, so that specific configuration is still a future option rather than something we can benchmark ourselves right now.

New Mac configurations for local LLMs

Q4Why does unified memory make Macs so useful for local LLMs?

Apple's unified memory gives Macs a real local-LLM advantage because the GPU can work directly with a large pool of memory that is also available to the CPU.

Apple's MLX documentation describes this very plainly: arrays live in shared memory, and CPU or GPU operations can work on them without first copying those arrays between separate memory pools.

A conventional PC often has a different bottleneck. We might install 128GB of normal RAM, yet an RTX GPU with 24GB of VRAM still cannot keep a 60GB model entirely inside its fast GPU memory. There are ways around that problem, including splitting or offloading the model, but they add complexity and can reduce performance.

A 128GB M5 Max gives the GPU access to a far larger memory pool inside one machine. That is why Macs become particularly interesting as model size rises.

Bandwidth then affects how quickly those weights can be moved during inference. Current Apple chips range from 60GB/s on the MacBook Neo to 614GB/s on M5 Max, with the new M5 Ultra reaching 1.2TB/s. Two Macs can therefore have enough memory for the same model and still deliver very different speeds.

Q5How much Mac memory do you really need for a local LLM?

For local LLMs on a Mac today, 16GB is a sensible minimum, 32GB is much more comfortable, and 64GB or more opens up a noticeably different class of models.

Quantization is what makes these numbers possible. Instead of storing every model weight at 16-bit precision, local versions commonly reduce weights to 8, 6 or 4 bits. We give up some numerical precision in exchange for a much smaller model.

OpenAI's gpt-oss models provide a useful reference point. OpenAI says gpt-oss-20b can run with 16GB of memory, while the much larger gpt-oss-120b fits within 80GB because its weights are distributed in the efficient MXFP4 format.

More recent models show how quickly the requirement is changing. In one current MLX coding benchmark on an M5 Max, a quantized Qwen 3.6 27B used about 17GiB. Qwen 3.6 35B-A3B used roughly 20.7GiB.

We still want breathing room above the model itself. macOS consumes memory, long contexts increase memory use, and coding tools or browsers may stay open beside the model. Buying a 32GB Mac for a model that regularly consumes almost 30GB would feel much tighter than the specification suggests.

How unified memory changes local LLM use

Q6Is 16GB enough to run a useful local LLM on a Mac today?

A 16GB Mac can currently run useful local LLMs, although we have to stay disciplined about model size and context length.

OpenAI designed gpt-oss-20b specifically to fit in 16GB of memory. It has 21 billion total parameters but activates only 3.6 billion for each token because it uses a mixture-of-experts architecture. OpenAI describes the model as suitable for local inference and edge devices.

Smaller models from Qwen, Gemma and other families also sit comfortably within this range after quantization. They can handle writing, summarization, structured extraction, basic coding and many private-document tasks without needing a cloud API.

Sixteen gigabytes becomes frustrating when we want to keep a larger model loaded while also giving it a very long context or running several heavyweight applications. We can absolutely do real work at 16GB these days; 24GB or 32GB gives us much more freedom.

Q7Is 32GB the sweet spot for running local LLMs on a Mac?

For most people buying a Mac specifically with local LLMs in mind, 32GB is currently the best balance between useful model size and avoiding workstation-level hardware.

A recent Apple Silicon coding benchmark makes that range concrete. Qwen 3.6 27B occupied about 17GiB in the tested quantization, leaving meaningful headroom inside 32GB. Qwen 3.6 35B-A3B used about 20.7GiB.

The same benchmark measured the 27B model at 32 generated tokens per second on an M5 Max. The 35B-A3B model reached 133 tokens per second because only around 3B parameters are active for each token.

Those speed results come from an M5 Max, so we should not expect an M6 Mac mini or M5 MacBook Air to produce identical numbers. The memory figures are still useful for understanding what can fit.

A few years ago, 32GB still sounded modest for local AI. Now it can hold models capable of serious coding, document work and local assistants rather than just lightweight chatbot demos.

Q8What changes when a Mac has 64GB or 128GB of memory?

A 64GB or 128GB Mac gives us access to local LLMs that simply cannot fit comfortably on ordinary consumer GPUs.

The 128GB M5 Max already gives us a good real-world example. Recent oMLX community testing loaded a 4-bit Qwen3.5 122B-A10B model on an M5 Max. At an 8K context, a recent run generated about 49 tokens per second while using roughly 70GB of peak memory.

Longer context changed the picture substantially. Another controlled run of the same model measured roughly 43 tokens per second at 32K context, 31 at 64K and 19 at 128K, while peak memory rose to about 86GB.

That is the useful way to think about 128GB: enough room for both a huge model and a genuinely large context, not merely enough memory to make the model file load.

At 64GB, we have less room for something as large as that 122B model, but plenty of space for strong models in lower weight classes, larger quantizations and demanding agent workflows. For someone who plans to use local AI heavily every day, 64GB is where memory constraints start disappearing from many normal workflows.

Q9Can the new 512GB Mac Studio really run enormous LLMs locally?

The new 512GB M5 Ultra Mac Studio should be able to hold enormous local LLMs with hundreds of billions of parameters, although its real-world speed is still unproven.

Apple has just announced the M5 Ultra with up to 512GB of unified memory and 1.2TB/s of bandwidth. The company explicitly says this allows huge LLMs containing hundreds of billions of parameters to run entirely on the device.

The capacity claim is straightforward. A quantized model occupying several hundred gigabytes can fit inside a 512GB memory pool in a way that would require several high-memory GPUs in a conventional workstation.

Apple is also adding RDMA support over Thunderbolt 5 so several Mac Studios can be clustered. According to Apple's own preproduction testing, a four-Mac Studio cluster reached up to three times the AI inference performance of one machine.

We should keep the performance claims in the right category for now. Apple measured the new M5 Ultra on preproduction hardware, and the new Studio has not reached normal buyers yet. The 512GB configuration comes later still. There are currently no broad independent M5 Ultra LLM benchmarks confirming Apple's speed claims.

The memory breakthrough itself is already clear. Half a terabyte of GPU-accessible unified memory in one compact desktop is a highly unusual setup for local AI.

Q10How fast do local LLMs actually run on the latest Macs?

Local LLMs on a high-end Mac can currently generate anywhere from a few dozen to well above 100 tokens per second, depending heavily on the model architecture.

A recent MLX coding benchmark on a 128GB M5 Max measured Qwen 3.6 27B at 32 tokens per second. Qwen 3.6 35B-A3B reached 133, while Qwen3-Coder-30B-A3B reached 129.

Thirty tokens per second already feels fast in a normal chat window. At more than 100, generation usually outruns the speed at which someone can comfortably read the answer.

The huge variation comes partly from how many parameters each model activates. A dense 27B model has to work through roughly 27 billion parameters for each token. The 35B-A3B model has 35 billion parameters stored in memory but activates only around 3 billion at a time.

Inference software can move the numbers too. In a separate controlled cross-platform test, the same M5 Max produced very different throughput depending on whether researchers used MLX, llama.cpp or Ollama.

So there is no honest single answer to “how many tokens per second does an M5 Max get?” Model architecture, quantization, context and software can change the result dramatically.

Q11Why are MoE models making Macs much better local AI machines?

Mixture-of-experts models are making local AI on Macs much more practical because they can provide a large total model while using only a small part of it to generate each token.

Qwen3.5 35B-A3B illustrates the idea well. The model contains around 35 billion parameters, yet only about 3 billion are active per token. Qwen3.5 122B-A10B stores roughly 122 billion while activating around 10 billion.

OpenAI uses the same basic idea with gpt-oss. Its 120B model contains 117 billion parameters but activates just 5.1 billion per token. The smaller 20B version activates 3.6 billion.

That combination suits Apple's architecture unusually well. A high-memory Mac can keep all the experts available in unified memory, while inference only needs to activate a fraction of them for each token.

The result is already visible in benchmark data. In one controlled M5 Max test using the same llama.cpp setup, Qwen3.5 35B-A3B generated 94 tokens per second, compared with about 25 for the smaller 27B dense model.

Parameter count by itself has therefore become a poor shortcut for predicting local-LLM speed. When we compare models for a Mac today, active parameters and architecture deserve almost as much attention as the headline model size.

Q12Are Macs actually faster than Nvidia GPUs for local LLMs?

High-end Nvidia GPUs are still faster than Macs on many local LLMs that fit entirely inside their VRAM, while high-memory Macs can run models that ordinary GPU setups cannot hold.

A useful controlled benchmark compared a 128GB M5 Max with two RTX 3090s providing 48GB of total VRAM. The researchers used the same llama.cpp engine and the same Q4 model format on both systems.

The dual RTX 3090 system generated about 118 tokens per second on Qwen3.5 9B versus 76 on the M5 Max. On the dense 27B model, the Nvidia machine reached 41 versus 25. With the 35B-A3B MoE model, it reached 139 versus 94.

Then the researchers loaded Qwen3.5 122B-A10B. The 128GB Mac generated about 43 tokens per second. The dual-3090 machine ran out of VRAM.

That result captures the Mac's current appeal. Nvidia still has enormous raw-compute advantages, especially for training and high-throughput serving. Apple's advantage shows up when one user wants a very large model on one quiet machine without building a multi-GPU workstation.

M5 Max versus dual RTX 3090 performance on Qwen3.5

Q13Can you run ChatGPT, Claude or Gemini locally on a Mac?

A powerful Mac still cannot run the actual proprietary models behind ChatGPT, Claude or Gemini because those companies do not provide their frontier model weights for local use.

What we can run are downloadable models such as Qwen, Gemma, DeepSeek and OpenAI's gpt-oss. LM Studio and Ollama make these models feel similar to using a cloud chatbot, but the model generating the answer is different.

OpenAI's gpt-oss is an especially useful example because the name can be confusing. OpenAI released gpt-oss-20b and gpt-oss-120b as downloadable open-weight models. They are separate from the proprietary frontier models served through ChatGPT.

So a 128GB or 512GB Mac can reduce how often we need ChatGPT or Claude, sometimes dramatically. It does not give us a downloadable copy of whatever model currently powers those services.

Q14Are local LLMs on Macs actually good enough for real work today?

Good local LLMs are already capable enough for real coding, writing, document processing and automation work, although the best cloud models still win on many difficult tasks.

The interesting change lately is how much capability fits into relatively modest model sizes. A current community coding benchmark tested Qwen 3.6 27B on 81 tasks covering coding, reasoning, tool use, math and writing. It scored 86.2% in that particular test suite. The benchmark's reported Claude Opus 4.6 API baseline scored 86.7%.

We should not turn one 81-task community benchmark into a claim that Qwen 27B has caught Claude overall. Claude has a much broader evaluation record, and proprietary frontier systems remain stronger in many complex agentic and multimodal tasks.

Still, the benchmark says something useful: on some real workloads, a local 27B model can now get close enough that choosing local no longer automatically means accepting a weak assistant.

For repetitive coding, transforming files, extracting structured information, querying private documents or running narrowly defined agents, we often care more about whether the model clears the required quality threshold than whether it ranks first across every benchmark.

And lately, that threshold is getting easier to clear.

Q15Do long contexts and coding agents need much more Mac memory?

Long-context local LLMs and coding agents can consume far more Mac memory than a simple chatbot, so users planning serious agent workflows should leave plenty of headroom.

The model weights remain roughly fixed once loaded. The growing part is the information the model has to keep track of while the conversation or task expands.

A coding agent may read source files, inspect terminal output, call tools, receive those results, modify code and then continue reasoning from the accumulated history. A large repository can push the context into tens of thousands of tokens surprisingly quickly.

The Qwen3.5 122B oMLX measurements show this directly. On an M5 Max, the tested 4-bit model used around 66GB of peak memory at a 1K context. At 128K context, peak memory had climbed to roughly 86GB, while generation speed fell from around 66 tokens per second to about 19.

This is why a 32GB machine with a 20GB model can feel perfectly comfortable for chat yet much tighter once we turn that same model into an all-day coding agent.

Anyone buying a Mac mainly for local agents should be more generous with memory than someone who only wants occasional local conversations.

Q16Does running an LLM locally on a Mac actually improve privacy and reduce costs?

Running an LLM entirely on a Mac can give us much stronger data privacy and remove per-token inference charges, especially for workloads we repeat constantly.

LM Studio can operate offline once the model files are present. In that setup, prompts and documents can stay on the computer instead of being sent to a remote model provider.

That is useful for private source code, internal company files, customer documents and personal data. We also control exactly which model version is running instead of having a hosted provider update it without our involvement.

Local inference can also make heavy experimentation much cheaper at the margin. Once we own the computer and have downloaded the weights, processing another document or generating another million tokens does not create an API invoice.

The economics depend heavily on usage. Someone who occasionally asks a frontier model a difficult question will usually spend less by paying for a cloud service. Someone running a coding assistant for hours every day, processing thousands of private documents or experimenting continuously has a much stronger reason to own the compute.

Privacy has the same qualification. The model can stay local while another part of an agent still sends information to web services or external APIs. A genuinely private setup requires the surrounding workflow to remain local too.

Q17Which Mac should you buy if you mainly want to run local LLMs?

For local LLMs right now, we would target 32GB for mainstream use, 64GB for serious daily AI work and 128GB when large models are a major reason for buying the machine.

An 8GB MacBook Neo would be our weakest choice for this purpose. Small local models will run, but memory will become the constraint very quickly.

A 24GB or 32GB MacBook Air or Mac mini is far more attractive for someone who wants local writing, document processing, coding help and experimentation without spending workstation money. Thirty-two gigabytes gives us access to a surprisingly strong set of models these days.

Sixty-four gigabytes makes sense once local agents, larger dense models or long context become routine. We can keep a substantial model loaded while leaving much more space for development tools and the operating system.

A 128GB M5 Max is the configuration we would choose if running genuinely large local models is one of the main jobs of the computer. Existing independent results already show 100B-plus MoE models running at interactive speeds on that machine.

The newly announced M5 Ultra Mac Studio sits in a much more specialized category. Its 256GB and 512GB options make sense for researchers, developers or teams who know they need models that consume well above 100GB. For everyone else, model efficiency is improving quickly enough that buying half a terabyte of memory for vague “future-proofing” would be hard to justify.

Q18So, can you easily run a local LLM on the new Macs?

Yes, running a useful local LLM on a new Mac is genuinely easy now, provided we choose enough unified memory for the models we actually want to use.

The software part is straightforward. Applications such as LM Studio can take us from installation to a working private chatbot through a graphical interface, while Ollama and MLX make it easy to connect local models to developer tools and agents.

The hardware question has also become much clearer. Eight gigabytes is restrictive. Sixteen gigabytes is workable. Around 24–32GB, local LLMs become useful for a broad range of normal tasks. At 64GB, we can use considerably larger models and heavier agents. A 128GB M5 Max already runs some 100B-plus models at interactive speeds.

Apple's latest move pushes the ceiling much higher. The M5 Ultra Mac Studio will offer up to 512GB of unified memory and 1.2TB/s of bandwidth, enough capacity for open models containing hundreds of billions of parameters. We should wait for independent testing before judging its real speed, but fitting these models on one desktop is already remarkable.

Nvidia remains faster for many models that fit comfortably in GPU VRAM. Macs have become compelling for a different reason: Apple lets us buy a very large pool of fast, GPU-accessible memory inside one compact machine.

So if someone asks whether they can download a good local model onto a new 32GB Mac and start using it without much trouble, the answer today is clearly yes. With 64GB or 128GB, local AI becomes much more serious. And at the very top of Apple's new lineup, the question is increasingly about how large a model we want to run rather than whether a Mac can run one at all.