Local 27B AI model: which Mac or PC can actually run one
A local 27B AI model on Mac or PC: how much memory it needs, what sets the speed, which apps run it and how to get the model when Hugging Face is blocked.
An open language model with 27 billion parameters weighs 14–17 GB once compressed. To run it properly you need an Apple Silicon Mac with 32 GB of memory or a PC with a 24 GB graphics card. A Mac with 24 GB or a 16 GB graphics card runs a more heavily compressed version, and on a 16 GB Mac a model of this class does not really fit. The models themselves are free to download from Hugging Face and run in free apps such as LM Studio and Ollama. Below: where these numbers come from, what determines response speed, which hardware works, and how models reach computers where Hugging Face is closed or the internet gets shut down.
Why 27B models became the core of local AI
Model size is measured in parameters, the numbers a model learns during training. More parameters means a smarter model and more memory.
Models with 7–14 billion parameters run fast even on a laptop but get lost in long reasoning. Models with 70 billion and up are noticeably smarter, but even compressed they weigh 40 GB or more. That means two expensive graphics cards or a Mac with a very large amount of memory.
The 24–32B class turned out to be the middle ground: the strongest models that still fit on a single 24 GB gaming card or a mid-range Mac. That is why several major labs compete in this weight class.
| Model | Developer | Released | Parameters | License | Understands images |
|---|---|---|---|---|---|
| Gemma 3 27B | March 2025 | 27B | Gemma Terms of Use | yes | |
| Qwen3-32B | Alibaba | April 2025 | 32B | Apache 2.0 | no |
| Mistral Small 3.1 | Mistral AI | March 2025 | 24B | Apache 2.0 | yes |
Apache 2.0 allows use anywhere, including in commercial products. Gemma comes with Google’s own terms: commercial use is allowed too, but with a list of prohibited uses.
New models in this class arrive every few months. Everything below about memory and speed applies to any of them: the rules are the same.
How much memory does a 27 billion parameter model need
In its original form each parameter takes 2 bytes. 27 billion parameters times 2 bytes is 54 GB. No gaming graphics card can hold that.
So models get compressed. The numbers inside are rounded so that each takes 8, 6, 5 or 4 bits instead of 16. The model becomes 3–4 times lighter. At 4–6 bits the answers barely differ from the original; at 3 bits and below mistakes on hard tasks become more visible.
In April 2025 Google released Gemma 3 versions that were trained in advance to work in compressed form. According to Google, Gemma 3 27B shrank from 54 GB to 14.1 GB in that variant.
Compressed models are usually downloaded as a single file in GGUF format. The compression level is written right in the name: Q8, Q6, Q5, Q4, Q3, Q2. The number after Q roughly equals the bits per parameter. Sizes in the table are calculated for a 27 billion parameter model; actual files differ by 1–2 GB.
| Compression level in file name | Bits per parameter, approx. | Size for 27B, calculated | Quality |
|---|---|---|---|
| original (BF16) | 16 | 54 GB | reference |
| Q8 | 8.5 | 29 GB | practically lossless |
| Q6 | 6.6 | 22 GB | difference hard to notice |
| Q5 | 5.7 | 19 GB | very close to original |
| Q4 | 4.8 | 16 GB | the most popular compromise |
| Q3 | 3.9 | 13 GB | more visible mistakes on hard tasks |
| Q2 | 3.1 | 10.5 GB | quality drops noticeably |
File size is not all the memory required. During a conversation the model keeps the context in memory: everything said so far plus any pasted documents. That part grows with the dialogue and with long documents takes several gigabytes. So you need headroom beyond the file size, around 2–3 GB for short chats (estimate) and more for long ones.
Which Mac can run a 27B model
Apple Silicon Macs have no separate video memory: the CPU and graphics share one pool. For local AI this is the main advantage: a 64 GB Mac can give a model more memory than NVIDIA’s biggest gaming card, the RTX 5090 with its 32 GB.
But macOS does not hand all of it to graphics. Part stays with the system and apps. By default graphics gets roughly two thirds to three quarters of memory (estimate, the share depends on the Mac model and macOS version). So a 24 GB Mac behaves like a 16–18 GB graphics card, not a 24 GB one.
| Mac memory | Available to graphics, estimate | What fits from the 27B class |
|---|---|---|
| 16 GB | 10–12 GB | does not really fit: only Q2 at about 10.5 GB, no headroom and a noticeable quality loss |
| 24 GB | 16–18 GB | Q3 with headroom, Q4 just barely, no room for a long conversation |
| 32 GB | 21–24 GB | Q4 with room for context |
| 48 GB | 32–36 GB | Q5 and Q6, long context |
| 64 GB and up | 42–48 GB and up | Q8, near-original quality |
For a 16 GB Mac the honest ceiling is models with 7–14 billion parameters. There are plenty of those, but it is a different class.
Apple also has its own AI library, MLX. Models in MLX format run natively on Apple Silicon. For popular models Hugging Face usually hosts both versions, GGUF and MLX, and LM Studio on a Mac can run either.
What determines response speed
A model writes its answer in small pieces called tokens. A token is roughly part of a word. To produce each next token, the computer reads the whole model from memory. So speed depends less on chip power and more on how many gigabytes per second the memory can deliver. That is called memory bandwidth.
This gives a simple upper bound: bandwidth divided by model size. Real speed is below that ceiling, but it shows the ratio between devices. Bandwidth in the table comes from Apple and NVIDIA specifications; the ceiling is calculated for a 16 GB Q4 model.
| Device | Bandwidth | Ceiling, tokens per second, calculated |
|---|---|---|
| Mac with M4 | 120 GB/s | 7.5 |
| Mac with M5 | 153 GB/s | 9.6 |
| Mac with M4 Pro | 273 GB/s | 17 |
| Mac with M4 Max, top version | 546 GB/s | 34 |
| Mac Studio with M3 Ultra | 819 GB/s | 51 |
| RTX 3090 | 936 GB/s | 58.5 |
| RTX 4090 | 1008 GB/s | 63 |
| RTX 5090 | 1792 GB/s | 112 |
For comparison, a person reads about 4–6 tokens per second (estimate). Anything above 10–15 feels like a fast answer. By this calculation the base M4 and M5 run a 27B model at the edge of comfortable, while M4 Pro and above handle it confidently.
Why a long document takes longer to answer
Before writing an answer, the model reads everything it was given. This stage works differently: what matters here is the graphics compute power, not memory. A short question is processed almost instantly on any device in the table. A long contract or a chat history dozens of pages long means a pause before the first line, and its length depends on how powerful the graphics is.
Apple indirectly acknowledges this weak spot: the M5 adds dedicated AI accelerators to every graphics core, and the company says the M5 delivers more than 4 times the peak GPU compute for AI of the M4. The exact gap in waiting time between a specific Mac and a specific graphics card depends on the model and the length of the text; there is no universal number.
Which graphics card does a PC need
On a PC, video memory decides everything. The part of the model that fits in the graphics card runs fast. The part pushed into regular RAM slows the whole process down, because ordinary PC memory delivers data several times slower than video memory.
| VRAM | Cards | What fits from the 27B class |
|---|---|---|
| 32 GB | RTX 5090 | Q5 and Q6, long context |
| 24 GB | RTX 3090, RTX 4090, Radeon RX 7900 XTX | full Q4, with about 8 GB left for context |
| 16 GB | RTX 4060 Ti 16 GB, 4070 Ti Super, 4080, 5070 Ti, 5080 | Q3 with about 3 GB of headroom, Q4 only partially |
| 12 GB or less | most mainstream cards | most of the model goes to system RAM |
The comparison with Macs lines up: a 16 GB card is roughly equal to a 24 GB Mac, a 24 GB card roughly equal to a 32 GB Mac. The difference is speed, not what fits.
Without a graphics card the model still runs on the CPU alone if the machine has 32 GB of RAM or more. That is enough to try it, not to work with it: ordinary PC memory delivers tens, not hundreds, of gigabytes per second, and the answer appears slower than a person reads. AMD cards are supported in LM Studio and Ollama, but most guides and ready-made setups are written for NVIDIA first.
Which apps run local models
Running a model stopped being a sysadmin job a while ago. The main apps are free.
| App | Platforms | Who it is for |
|---|---|---|
| LM Studio | Mac, Windows, Linux | a regular app with a chat window, point-and-click model search and download |
| Ollama | Mac, Windows, Linux | runs in the background, handy when other programs need the model |
| llama.cpp | Mac, Windows, Linux | the engine LM Studio and Ollama are built on, for command-line fans |
In LM Studio the whole path is point and click: search a model by name, pick a compression level, download, chat. The app itself highlights whether the chosen file will fit in your computer’s memory, so there is no need to count gigabytes from the tables above by hand.
LM Studio and Ollama have a mode in which the model answers other programs the same way ChatGPT does. A bot or script written for ChatGPT switches to your own model by changing an address in its settings. That is how a local model becomes part of a product rather than just a chat window.
On the Mac, ready-made tools are appearing on top of this. KRAKOBIT has already covered Osaurus, which runs AI agents on a Mac without internet. The 27B class is exactly the size at which such agents start handling multi-step tasks rather than just keeping up a conversation.
How to download a model if Hugging Face is blocked
Hugging Face is the main warehouse of open models, and it is blocked in China. Chinese developers use two routes. The first is hf-mirror.com, a community mirror that copies Hugging Face content. The second is ModelScope, Alibaba’s platform, which hosts Qwen and many other open models.
In Iran the main problem is shutdowns. In November 2019 the authorities cut the country off from the international internet for almost a week. In June 2025, during the war with Israel, connectivity to the outside world nearly vanished again. During such shutdowns nothing can be downloaded from abroad, while a model already on disk keeps working.
A 16 GB file takes a long time over a slow or unstable connection. Calculated without drops and losses:
| Connection speed | Time to download 16 GB, calculated |
|---|---|
| 2 Mbit/s | about 18 hours |
| 10 Mbit/s | about 3.5 hours |
| 50 Mbit/s | about 43 minutes |
| 100 Mbit/s | about 21 minutes |
GGUF is convenient because the model is a single file. It can be copied to a USB stick or external drive and handed to someone without internet. LM Studio and Ollama pick up such files from disk with no network download.
Copies from mirrors and chats carry an obvious risk: anyone could have prepared the file. The page of every file on Hugging Face lists its digital fingerprint, a long string of letters and numbers. If the fingerprint of your copy matches the listed one, the file is identical down to the last byte.
What a local model can and cannot do
Works well:
- texts, correspondence, translation, summarizing documents;
- help with code at the level of single functions and bug hunting;
- fully offline work: on a plane, on the road, during shutdowns;
- privacy: contracts, emails and customer databases never go to someone else’s server;
- independence from service rules: cloud AI depends on whether the company lets in users from your country and accepts your payment, while a downloaded model does not care where the computer is;
- answers for your own bots and agents without a bill per token.
Works worse or not at all:
- it does not know recent news: its knowledge ends at the training date and it does not go online by itself;
- on long multi-step tasks models of this class trail cloud flagships such as Claude and GPT;
- long context eats memory and lengthens the pause before the answer;
- not every 27B-class model understands images, the model description says which do.
What your own model costs compared with a subscription
ChatGPT Plus and Claude Pro cost 20 dollars a month each, 240 dollars a year. For that the user gets the strongest models, along with message limits and ties to an account. The API charges for every token, and agents that take dozens of steps per task burn through thousands of them.
A local model has different economics: money goes into hardware once rather than into words every month. If a 32 GB Mac or a 24 GB graphics card is already on the desk, a 27B-class model adds only electricity to the bill. Without the hardware the entry cost is higher: a used 24 GB RTX 3090, for example, costs roughly 600–900 dollars (estimate, prices vary a lot by country), which is 2.5–4 years of a subscription.
In practice the two are combined more often than one is chosen over the other. Simple agent steps such as sorting email, triaging requests and drafting go to the local model, while the paid cloud model is kept for hard tasks. Under that setup the API bill depends on how much of the work can be moved to your own computer.
Summary: what runs on which hardware
| Hardware | What actually runs from the 27B class |
|---|---|
| Mac 16 GB | does not really fit, only Q2 with quality loss; ceiling is 7–14B models |
| Mac 24 GB | Q3, Q4 just barely |
| Mac 32 GB | Q4 with room for context |
| Mac 48–64 GB | Q5–Q8, long context |
| PC with a 32 GB card | Q5 and Q6 fully on the graphics card |
| PC with a 24 GB card | Q4 fully on the graphics card |
| PC with a 16 GB card | Q3, or Q4 with partial offload to system RAM |
| PC without a graphics card | runs for testing, slowly |