Maybe the fact I have a family and child to support nowadays, but no matter how much I have in the bank I personally don't feel I could justify spending the extra 3-4k to get a few extra tokens a second. If you're absolutely dedicated on getting that extra 1-2 tokens a second extra and willing to pay double or more to get it via an M2 ultra then fair enough, just don't try to justify the purchase to anyone else. Also less likely to loose as much on resell ( as it's had its first major drop already with M2 competition ). Good t/s, known capabilities, and a price that is much more fair and accessible. Some of the best advice I've seen in this sub so far. The M1 Ultra has 800 GB/s memory bandwidth and the M2 Ultra also has 800GB/s memory bandwidth I dunno, maybe it's because I'm spoiled by fine-tunes, but the answers are just so 'meh' compared to the 70b models lol Just my personal opinion.ĮDIT 2: The inference is likely identical because they both have the same memory Bandwidth. I regularly run 70b q8 models and they are so fantastic that I can't even describe the M1 Ultra can run them just as well. Here's an old post from on LocalLlama with some M1 inference numbers:ĮDIT: In case you were curious, I haven't found the 5_K_M 180b Falcon to be worth it lol. Just jam an external drive on it for cheaper than buying more storage that's what I did with my M2 since the models are loaded into RAM anyway. You can buy the M1 Ultra Mac Studio for $3700 refurbished from apple if you get a 1TB hard drive. The only difference between the two is that the M1 can run a 3_K_M of the 180b while I can run a 5_K_M. The 128GB has 97GB of VRAM, while the M2 Ultra has 147GB of VRAM. I have an M2 Ultra Mac Studio with 192GB and have compared numbers against the M1 Ultra Mac Studio with 128GB. Should I bite the bullet and get Mac Studio M2 Ultra ($7000 after tax)
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |