> M5 Ultra features a massive amount of high-bandwidth unified memory, up to 512GB, and delivers a staggering 1.2TB/s of unified memory bandwidth that is 50 percent higher than M3 Ultra.
Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.
As someone who works in AI now, I have found it pretty amazing that Apple basically didn't do much with AI software, and focused more on the hardware side. I think this is what the future of AI is going to look like, local models run on your mac for your workflow.
It's possible that they're working on their own LLM that's going to work very well on their chips, and possibly outperform anything out there when they do release it.
10 years ago 32GB ram laptops sounded too much. 8 was enough. These days even I would get that much ram since it’s soldered. 64GB is higher end.
In a few years we should see such high end hardware commonplace. Working with a local LLM to get work done is the ideal way to go which has mostly hardware limitation as of now that gets solved in due time.
> 10 years ago 32GB ram laptops sounded too much. 8 was enough. These days even I would get that much ram since it’s soldered. 64GB is higher end.
Ten years ago I got 64gb of ram in my laptop, same as I have now. I bought both for business and personal use. System ram capacity hasn't changed much in 10 years.
It makes me curious how old you were 10 years ago.
We were definitely outliers that long ago. I put 64 GB in a MacBook Pro back in 2019, and that was (a) overkill for everything I ever ran on that machine, and (b) stupidly expensive by 2019 standards (albeit almost affordable by 2026 standards)
>I have found it pretty amazing that Apple basically didn't do much with AI software
The iPhone 15 was almost entirely marketed based upon AI (I would say fraudulently so, advertising features they still haven't delivered), and a huge portion of the OS work was on local AI or AI integration.
And for that matter Apple has been dumping enormous sums into their own AI development. Their failure to have a lot to show for it doesn't void the fact that they tried really, really hard.
It's bizarre how often this "Apple sat on the sidelines and let the AI people fight...so smart!" narrative appears on HN. Apple hasn't gone down the path of spending hundreds of billions on nvidia GPU data centres, but they absolutely tried really hard to matter in AI.
Is there anything comparable that runs Linux, doesn't necessarily look as good, but is perhaps (a lot) cheaper/fixable? Or is this really pretty optimal?
I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps?
I want to get something for my company to run local models, wondering what would be a good option.
I love linux and would be using it if the ARM support was better. It's just not there and most distros that support ARM do it a little poorly. I just haven't seen anything even remotely comparable to Apple Silicon and unfortunately Linux is struggling very hard to support it.
It's not quite that ARM support isn't good on Linux, it's that there aren't high-performance ARM chips with strong general-purpose software stacks. Like the Raspberry Pi is very well supported, but otherwise the only upmarket devices are things like Ampere workstations and hyperscaler server chips.
The PC platforms have anemic memory bandwidth in comparison. Eg, Strix Halo is 256GB/s max. If money is a bigger limiter than performance it can be an option though. As can Nvidia DGX Spark machines. (Also limited to 128GB memory and comparatively low bandwidth, but higher compute than Strix Halo.)
AFAIK, apple does not release drivers open source, asahi is a reverse-engineering endeavour and does not support GPU.
For nvidia, there are both proprietary and open-source linux drivers. CUDA and inference works on linux with nvidia.
I would recommend checking out this video of Alex Ziskind to shop for a computer to run local LLMs: https://www.youtube.com/watch?v=mevUEQcumzU&t=224s.
TL;DR besides Apple he recommends, DGX Spark, Tenstorrent Wormhole N300, AMD Radeon 7900 and NVIDIA RTX 5090.
I guess, what I mean is: Why are these tiny aluminum boxes so optimal?
I just want my butt ugly repairable beast machine to do the same trick. Why is my ram not unified? I have an iGPU in my server, but it can't access the 64 GB ram (I got last year for 150 euro) directly or something? It's on the CPU right? Why did only Apple go for this architecture? So many questions...
well the good news is you can indeed already just have unified cpu/gpu memory on linux with an igpu on good old replaceable ram. i've done it on 8th gen intel stuff i picked up dirt cheap used. for the most part, running on the gpu wasn't faster (nor appreciably slower) than the cpu for the things i was doing, just more power efficient. overall bandwidth is relatively limited regardless which would be the bigger difference comparing against the m chips. and of course, good luck if you're hoping stuff like opencl support hasn't long ago been ripped out of whatever software you might perfectly reasonably expect to run this way today
You can't run Linux directly on these. Asahi Linux supports up to M2 only.
Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels).
But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't.
However, macOS on M-series macs is excellent for local models. (Maybe not as excellent as a box full of the best nVidia GPUs, but still excellent).
So if you're getting Apple hardware, like Linux, and want to run all of it locally, a fine setup for a machine to run local models, with agentic characteristics:
- macOS running one of the many local model runners. I used to use Ollama and Whisper, and now use llama.cpp instead of Ollama. Others use LM Studio, oMLX, etc. Provide HTTP endpoints to access the models.
- Linux in a VM for overall control and orchestration, with standard VM settings, and bridged networking so it appears as its own machine on your network. Also, in here provide a robust shared file server for shared state. Use this VM as your desktop and primary access to the machine, if you like Linux.
- Linux in a VM to launch ephemeral, volatile containers, with the containers using a memory-only tmpfs overlay on top of a read-only Linux filesystem in a VM disk image, with tools in this filesystem. Alternatively, a writable Linux filesystem in a VM disk image, with disk buffering set to use macOS host buffering and discard fsync requests. These settings optimise for container disk performance for data that's only ephemeral which will be deleted soon or on system shutdown. (You can combined both VMs, but need to use two VM disks to get equivalent behaviour, and be careful about VM disk configuration of the two disks.)
- Containers spawned within that second Linux VM can be spawned very quickly and run quickly, so are ideal for LLM agents that need a quick sandbox. These sandboxes generally run faster than a macOS sandbox, despite being on the same machine with VM overhead, because Linux is faster at some things. Teach the LLMs to store files and memories they want to keep in the shared file server.
There is no magic, if the data you compute as atomic chunk don't fit in cache then memory bandwidth R/W limit kicks in and architecture does not matter. On contrary - having multi gpu setup of same price and same memory size with even slower memories may give you effectively much higher bandwidth but at the cost of power consumption.
Same here. Sadly I think the voices like ours won't be heard, though, because Apple's looking for someone who's going to buy in on the whole ecosystem, and I think we're not it. Or at least I'm not.
It's worth noting that the 5090 (or the RTX Pro 6000 big brother with 92GB VRAM) will run rings around the Mac when it comes to compute.
My old 3090 is typically significantly faster (almost 2x token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.
In most situations it's a better idea to just buy tokens. But there are definitely cases when that's not an option. And then a machine like the M5 Ultra can allow you to do things locally for a fairly limited budget. And in a simpler package to manage than a machine with multiple GPUs.
I was so blown away at all the discourse surrounding "Apple fumbling on models". They should never have been in the model game to begin with. Apple crushes hardware over the last decade and that's a huge advantage today. In the end, massive models have proven to be very strong, but small models have proven to be good enough (especially with the recent Qwen 2.8 27B drop) and that's where I imagine the future will lie for consumers.
I think this is a bit of a crazy statement. Everyone expects Apple to somehow build a category leading product every year. I'd expect something innovative every couple of years
* the iPhone
* the iPad
* apple watch
* airpods
* unified memory laptops and computers
Those are all products that either created a category or changed that industry.
I think that each of the products you name is the top or near the top of their category, but these weren't creating a category. I don't think my original comment says anything about "changing that industry", so that's a bit of a strawman. They absolutely change the industry they're in. They're just not first to any of those categories that you mentioned (maybe unified memory, I'm not sure).
They weren't the first smart phone, tablet, smart watch, or true wireless earbuds. They did a damn fine job making each of those though. I am typing this on a my work macbook wearing AirPods, and AppleWatch, listening to audio on my iPhone. Apple does a really good job with their products.
They did participate early on with Apple Intelligence and failed miserably. Really good move to not double down and let the others explore the space first
Probably too young if it was easy these companies would still be around I miss them SGI, Sun, Digital, Acorn, particularly SGI and Acorn. All were roadkill of Wintel.
Apple has the mlx framework. Most/all major software for running models locally support it. Apple also has RDMA for interconnecting multiple machines across Thunderbolt connections.
Uhhh, they are? They’re a hardware co for sure, but to say they aren’t focusing on on-device models is absurd on its face. They’ve spent over 2 years on Siri AI which is (mostly) local.
I'm not sure who that line is supposed to impress. Gamers focusing on graphically demanding AAA games would laugh at this. People who don't game much probably won't know whether this is good or not.
There are some older games (not that old, one of the metal gear/raiden games had it, so like 10-15 years) where the cut scenes are actually prerendered video playback - so not very demanding. Don't know if mixtape does this though.
Apple Studio with maxed out M5 Ultra, 256GB RAM and 16TB storage is 18,299$. The 512GB RAM version apparently is coming in October, considering that the difference between 96GB and 256GB is priced at 4000$, the 512GB upgrade must be eye watering.
So, on the mini the RAM upgrade runs at 25$ per GB on all tiers, the same as the Studio therefore the upgrade to 512 will probably cost 6400$.
The fully maxed out Apple Studio then will be 24699$. It's 17199$ if you don't upgrade the storage(1TB).
So is downpayment on a house. I would buy the house and just pay for tokens as needed. The house will get more valuable and that wealth would buy a lot of tokens in the future - which will probably get cheaper.
EDIT: or buy AAPL. If I had bought Apple stock instead of buying a Mac LC II in 1992, then I would have about $2 million in Apple stock.
I tripled my money on my RAM purchase of three years ago. So, yes, for short-term appreciation that's hard to beat. But I don't think it's something that will continue.
Or more simply – $25K (+ tax) put in a savings account will earn about enough interest to pay for a $100/month AI subscription indefinitely. And at the end of it you still have the $25K.
I don't need privacy, so it would be financially imprudent for me to spend 20 grand on such a machine. But I have a financial management client who does need such privacy, and if I get more fully engaged with them then I would be able to justify getting a loaded Mac.
Not 'more simply', there are basically zero savings accounts that are going to net you a 5%+ interest rate to give you that $100 a month. And that $25k becomes less valuable over time. $25k is now only worth $19k because inflation.
Do you think monthly AI 'subscriptions' are going to be $100 a month in 5 years? These people using these would probably be on $200/month subscriptions and with that OPs assertion of 'shoving away and paying with interest' makes no sense.
Currently m5 max has a prefill rate for Qwen 3.8 27B of 400+ tok/s, then generate at ~60 tok/s. (And it can be improved further, the software is not yet at the level it is for CUDA)
That means for context under ~5k or so ttft (time to first token) it's going to respond faster than Claude. If the answer is less than ~1k I think the request finishes sooner. And it's ~claude 4.5 or 4.6 level intelligence.
I've had it work for more than a day on a pi.dev "loop engineering" project involving writing software.
It’s basically impossible to compete on economic terms with deeply subsidized hardware that is widely available to rent or as a service with zero commitment.
For general inference there’s no ROI that makes this work vs subscriptions.
25k for computer now, plus 9-10% sales tax, plus operating cost, plus time and cost for R&D tinkering with models, harnesses, and infra (assuming highly capable engineering talent that can get paid for your human inference) vs a HEAVILY subsidized subscription at 200 per month with free R&D has a pretty long ROI (15 years?)
At API costs, it’s like 6 months if you’re heavy on inference.
For training, specialized models will have their own ROI that makes this worthwhile. Then debate renting capacity and the platform to choose
Yeah I think you call out the right incentives. Lots of people are very interested in renting out the hardware, and it gives us lots of flexibility vs buying.
If the pace of hardware change is what the jensens of the world say, then the rents on existing hardware will decline.
Why are we comparing to the maxed out model only? The M5 Ultra with 96 GB, 1 TB, and 64 GPU cores is $5499.
Apple lets you lease that model for $110/month for 36 months.
If you can settle for a M5 Max base model that would be a $49/month lease.
Today, you should be able to run Qwen 3.8 - 27B amazingly well on either which is giving comparable performance to 5.6 Luna on SWE Bench. The local models are now getting better and more efficient and this should give you headroom. Tools like turbo fieldfare are really reducing the memory requirements to run large models and I don’t see it stopping soon.
Maybe they had the cash on hand, maybe they didn’t. Assuming they are in the family formation stage of life with young kids, an outlier risk appetite would be required to sleep well at night.
Well, this is all assuming a 20% down payment, which anecdotally as a 26 yr. old, nobody I know is able to achieve. All my home-owning friends put 3-7% down. Granted, most are using first-time homebuyer loans which are generally more favorable.
Depends on where you live. Also keep in mind population decline so long term, im not so sure. Prices already dropping in less desirable places (everywhere outside of most blue areas) (Assuming you are talking about the US sorry if not)
Apple never needed to participate in the AI race to zero. Because they were already at the finish line years ago building their own chips that can run large >100B parameter AI models locally.
It's possible that they're working on their own LLM that's going to work very well on their chips, and possibly outperform anything out there when they do release it.
10 years ago 32GB ram laptops sounded too much. 8 was enough. These days even I would get that much ram since it’s soldered. 64GB is higher end.
In a few years we should see such high end hardware commonplace. Working with a local LLM to get work done is the ideal way to go which has mostly hardware limitation as of now that gets solved in due time.
Ten years ago I got 64gb of ram in my laptop, same as I have now. I bought both for business and personal use. System ram capacity hasn't changed much in 10 years.
It makes me curious how old you were 10 years ago.
We were definitely outliers that long ago. I put 64 GB in a MacBook Pro back in 2019, and that was (a) overkill for everything I ever ran on that machine, and (b) stupidly expensive by 2019 standards (albeit almost affordable by 2026 standards)
Yep, Siri AI; they’re doing it in public.
The iPhone 15 was almost entirely marketed based upon AI (I would say fraudulently so, advertising features they still haven't delivered), and a huge portion of the OS work was on local AI or AI integration.
And for that matter Apple has been dumping enormous sums into their own AI development. Their failure to have a lot to show for it doesn't void the fact that they tried really, really hard.
It's bizarre how often this "Apple sat on the sidelines and let the AI people fight...so smart!" narrative appears on HN. Apple hasn't gone down the path of spending hundreds of billions on nvidia GPU data centres, but they absolutely tried really hard to matter in AI.
I mean this is not nvidia based right? It's all custom? So we can use it under Asahi perhaps?
I want to get something for my company to run local models, wondering what would be a good option.
I just want my butt ugly repairable beast machine to do the same trick. Why is my ram not unified? I have an iGPU in my server, but it can't access the 64 GB ram (I got last year for 150 euro) directly or something? It's on the CPU right? Why did only Apple go for this architecture? So many questions...
Linux runs very well in a VM on macOS. There are many good options for this, some free and open source (QEMU, UTM, Lima, Colima), some proprietary (VMware Fusion, Parallels).
But Linux in a VM doesn't get access to the real GPU, so model performance is limited. Those running on the CPU perform well, and those needing the GPU don't.
However, macOS on M-series macs is excellent for local models. (Maybe not as excellent as a box full of the best nVidia GPUs, but still excellent).
So if you're getting Apple hardware, like Linux, and want to run all of it locally, a fine setup for a machine to run local models, with agentic characteristics:
- macOS running one of the many local model runners. I used to use Ollama and Whisper, and now use llama.cpp instead of Ollama. Others use LM Studio, oMLX, etc. Provide HTTP endpoints to access the models.
- Linux in a VM for overall control and orchestration, with standard VM settings, and bridged networking so it appears as its own machine on your network. Also, in here provide a robust shared file server for shared state. Use this VM as your desktop and primary access to the machine, if you like Linux.
- Linux in a VM to launch ephemeral, volatile containers, with the containers using a memory-only tmpfs overlay on top of a read-only Linux filesystem in a VM disk image, with tools in this filesystem. Alternatively, a writable Linux filesystem in a VM disk image, with disk buffering set to use macOS host buffering and discard fsync requests. These settings optimise for container disk performance for data that's only ephemeral which will be deleted soon or on system shutdown. (You can combined both VMs, but need to use two VM disks to get equivalent behaviour, and be careful about VM disk configuration of the two disks.)
- Containers spawned within that second Linux VM can be spawned very quickly and run quickly, so are ideal for LLM agents that need a quick sandbox. These sandboxes generally run faster than a macOS sandbox, despite being on the same machine with VM overhead, because Linux is faster at some things. Teach the LLMs to store files and memories they want to keep in the shared file server.
But you get a generic computer and much more RAM.
And you lose a couple of organs.
It would take Apple one or two engineers to make Linux life much easier on macs. But Linux is outside their walled garden so it's ignored.
My Mac Mini is strictly a headless server for llama.cpp.
I use a Linux workstation.
If I were limited to use Mac hardware , I would install Linux in VMware Fusion and work from there.
My old 3090 is typically significantly faster (almost 2x token/s) than my M4 Max 128GB machine, as long as the model fits in the 24GB of VRAM.
In most situations it's a better idea to just buy tokens. But there are definitely cases when that's not an option. And then a machine like the M5 Ultra can allow you to do things locally for a fairly limited budget. And in a simpler package to manage than a machine with multiple GPUs.
* the iPhone * the iPad * apple watch * airpods * unified memory laptops and computers
Those are all products that either created a category or changed that industry.
They weren't the first smart phone, tablet, smart watch, or true wireless earbuds. They did a damn fine job making each of those though. I am typing this on a my work macbook wearing AirPods, and AppleWatch, listening to audio on my iPhone. Apple does a really good job with their products.
Realizing how surrounding by Apple I am...
FWIW, Ollama, LM Studio and Lemonade (and oMLX) also wrap Apple's MLX framework.
It's the Hardware, Software and Services in combination. None would work without the other (to reach the scale apple is)
More specifically, they're a hardware dongle company first
not getting on that bandwagon but wasn't that not the most demanding game as its a just a nonstop cutscene.
It's a very deliberate choice when if it doesn't make sense to gamers.
* it's critically acclaimed (86 metacritic, 10/10 IGN)
* whatever person decided this likely knows nothing about video games
* most importantly it's a modern game in UE5 that's COMING NATIVELY TO MAC, including to the App Store
What would you have put?
In US its $4000 upgade so $25 for 1GB.
Also:
> 512GB memory option for M5 Ultra coming late October
New Hampshire, Oregon, Montana, Alaska, Delaware.
So, on the mini the RAM upgrade runs at 25$ per GB on all tiers, the same as the Studio therefore the upgrade to 512 will probably cost 6400$.
The fully maxed out Apple Studio then will be 24699$. It's 17199$ if you don't upgrade the storage(1TB).
Nevertheless I itch to have one :)
EDIT: or buy AAPL. If I had bought Apple stock instead of buying a Mac LC II in 1992, then I would have about $2 million in Apple stock.
Unless you need privacy for your inference this instant, paying for credits can get 80 to 90 percent of people everything they need.
Of course if you do need that privacy, then forking the $25K over to Apple is a no brainer.
There are both cheaper and faster options out there.
If treasuries “fail” we have a different class of problem.
???
30 Day SEC Yield as of Aug 24, 2026: 3.61%
12m Trailing Yield as of Aug 24, 2026: 3.74%
https://www.ishares.com/us/products/314116/ishares-0-3-month...
1.39% might not seem like a lot, but that's off by more than a quarter.
Our assumptions may differ - and that’s ok... The monthly costs may go down too.
That means for context under ~5k or so ttft (time to first token) it's going to respond faster than Claude. If the answer is less than ~1k I think the request finishes sooner. And it's ~claude 4.5 or 4.6 level intelligence.
I've had it work for more than a day on a pi.dev "loop engineering" project involving writing software.
Plus privacy. Plus offline.
For general inference there’s no ROI that makes this work vs subscriptions.
25k for computer now, plus 9-10% sales tax, plus operating cost, plus time and cost for R&D tinkering with models, harnesses, and infra (assuming highly capable engineering talent that can get paid for your human inference) vs a HEAVILY subsidized subscription at 200 per month with free R&D has a pretty long ROI (15 years?)
At API costs, it’s like 6 months if you’re heavy on inference. For training, specialized models will have their own ROI that makes this worthwhile. Then debate renting capacity and the platform to choose
If the pace of hardware change is what the jensens of the world say, then the rents on existing hardware will decline.
If you can settle for a M5 Max base model that would be a $49/month lease.
Today, you should be able to run Qwen 3.8 - 27B amazingly well on either which is giving comparable performance to 5.6 Luna on SWE Bench. The local models are now getting better and more efficient and this should give you headroom. Tools like turbo fieldfare are really reducing the memory requirements to run large models and I don’t see it stopping soon.
https://github.com/drumih/turbo-fieldfare
… I wish I hadn’t just calculated that.
Maybe they had the cash on hand, maybe they didn’t. Assuming they are in the family formation stage of life with young kids, an outlier risk appetite would be required to sleep well at night.
In RTP (NC), a ~$400k house at 5% down is $20k
You get what you pay for.
Depends on where you live. Also keep in mind population decline so long term, im not so sure. Prices already dropping in less desirable places (everywhere outside of most blue areas) (Assuming you are talking about the US sorry if not)
problem solved /s