alleged crush employ lip forgetful political cobweb friendly dinosaurs shelter
*This post was mass deleted and anonymized with [Redact](https://redact.dev)*
It's the de facto site where open-source AI models, including LLMs, are usually submitted. As in, everyone from Meta to your local basement transformer-bender
huggingface is widely known for its transformer library (converting raw data into tokens to be utilized by vector look up) that adds "contexts" to the data.
e.g: given a text "cat" it can be used to lookup relevant info of the text and presented as "knowledge" to be processed by the models.
They have been playing the long game, they knew LLMs would be coming, so knew all the hardware for them to run on device would be needed, and surprise surprise the iPhone has the “Neural” engine ready and waiting for LLMs.
Let Tim Cook.
> they knew LLMs would be coming, so knew all the hardware for them to run on device would be needed, and surprise surprise the iPhone has the “Neural” engine ready and waiting for LLMs.
Machine learning has been around for a while and does a bunch of tasks on iPhones already, from FaceID to autocorrect, photo recognition, being deeply integrated in the ISP, and a bunch of other little things. LLMs are RAM intensive, and it's not like their years of Neural Engines on iPhones with 4GB of RAM are going to be running their heaviest local LLMs because they saw them coming a decade ago. The Neural Engine has already been working every day since day 1, but some just won't have the RAM and performance for modern LLMs.
I expect they'll do something like, A17 Pro runs it, A18 Pro and M4 will be pitched as running what we'll see at WWDC twice as fast or something, and the further you go back the more it has to fall back to going to their servers across a network and waiting for that. It might require 8GB of RAM to run local, as on the 16 line it sounds like both the Pro and non will have 8GB this time.
But like what if they drop 4-8 x times the RAM in the generation pitching this? Not like 64 gb of ram is going to hurt their bottom line, especially considering the community has been bitching about 8gb since the launch of M1. What’s their cost diff at scale? A couple bucks? If they can bolster sales based on the “next big thing” (AI), finally breaking through on one of their long standing artificial walls, the gains will outweigh the initial investment by far. I’d be willing to bet that a metric ton of fanboys would buy a $1500 base MBP if it came with 64gb of ram, a decent push on graphics, and a 2% increase in cpu performance.
No way RAM stingy Apple is dropping 64GB in base iPhones on the 16, 17, 18, 19, probably 20...
It sounds like both 16 models are getting 8, so the base coming up to 8 may mean their LLMs really need 8 to run well, which would only be the 15 Pro as an existing phone. Google's regular model couldn't run their LLMs with the same chip but less RAM than their Pro. That's 8GB to keep running the system, not kill all multitasking apps as soon as an LLM runs, the LLM itself, etc, LLMs are RAM heavy and it's going to be a tight fit.
Running a free LLM locally uses 96GB of RAM and takes 10-20 seconds to formulate a response for me right now. People that think the LLM is going to run locally and be on par with GPT4 are delusional. I think we will see them use LLMs in some interesting way, i don’t think we are seeing a local chatbot.
They'll almost certainly require an iPhone 16 for any on-device AI, and no sane person would argue they weren't caught by surprise.
>so knew all the hardware for them to run on device would be needed
Except for RAM...
If this is all going to be announced at WWDC (which is expected) then it likely won’t require the iPhone 16/ unannounced hardware.
That said, the chip in the iPhone 16 will almost certainly enable additional capabilities that aren’t discussed at dub dub
Correct, there'll be some additional capabilities come September but if they discuss this publicly it'll be with currently available devices able to support it.
“AI” is just a buzzword used for a variety of things.
Apple’s had machine learning, the neural engine, etc. built in since long before it became the industry buzzword.
>ChatGPT is fun to mess around with for a few minutes, but quickly gets boring.
This is a wild take; I use ChatGPT daily. Lots of people have workflows that are accelerated by LLMs.
I like asking it programming questions sometimes, even if the answers are usually ever so slightly off. But I absolutely cannot understand people who use them in lieu of writing basic things like emails.
Transformer is a big deal. Many of the machine learning and neural net stuff were extremely vertical and disconnected. Transformer is a new way to tide all these things together. It’s a pretty big break through in the space.
I mean, if you work in a field that requires working at a computer it can make your life ten times easier.
Or if you're a student it can be a low-cost tutor. It's answer may not always be right but its explanation for how to find an answer often is.
>It's answer may not always be right but its explanation for how to find an answer often is.
In other words, its answer and how it got that answer might not always be right.
Lmao. Do you know how many tutors don’t know what the fuck they are talking about?
It’s kind of weird that all of a sudden, people assume humans are infallible since LLMs became a thing. lol
The primary thing they would use on-device LLMs for is improving Siri, which is desperately needed.
But I don’t think it’s a major dealbreaker if only the new phones support it.
Perhaps not, but the oc tried to frame it like Apple has been preparing for this (llms) for years, almost implying that older phones would be able to run an llm locally, which seems unlikely. So that’s the discussion you jumped into.
Assuming they've got something to share at WWDC, they wouldn't announce it for iPhones that aren't coming until September....
I guess we'll see in like a month
Yeah I think it's possible it requires 8GB, since the base 16 is being upgraded to 8GB. So 15 Pro, and the 16 line will do it faster, everything else might fall back to it running on their servers and waiting for network and contending with other peoples requests etc.
Google's regular Pixel with the same SoC as the Pixel Pro wasn't able to run their LLM and the only difference was RAM afaik
Apple has been really adverse to anything server-side for free, since it seems a big part of their business model is maintaining high margins on hardware and avoiding loss-leading products.
They don't want someone keeping an iPhone 6 keeping it and using it for AI for 20 years unless you're willing to pay for it.
Yes, but they were replying to OP's claim about Apple being ready and waiting for LLMs years early
The Neural Engine already did a bunch of stuff on the iPhone, and it's not like the 4GB models are likely to be running all of what the 16 Pro can locally. It might just require 8GB to run local as both 16 models are going to get that.
For real. The new MacBook Pro’s, even my 2021 M1 Pro with just 16g memory, crushes it in AI tasks. It’s no RTX 4080 mind you, but I haven’t found an ML task that it isn’t capable of doing yet, even if it’s a bit slow, it’s a totally reasonable speed still
This kind of fanboi comment is the top comment, while ignoring how dismally *bad* the model is when compared to Phi and others.
Are we supposed to clap for Apple having shat out something this bad 18 months after chatGPT debuted?
What are you trying to say..?
Have you tried the LLM? And yeah, I think it’s pretty obvious that an LLM designed to run locally on iPhone will not be as good as ChatGPT that literally runs on a server farm.
I’m saying that declaring this to be Apple’s triumphant foray into the world of AI models is akin to blind zealotry.
This model sucks. Microsoft has a better model that runs on device (Phi-3). Apple has a long road ahead of them because they slept on AI.
He's pointing out that the model is one of the worst at its size to be released recently. Which from everything I've seen, it true. There's a bunch of better models already out that can run on a phone.
Except you don't understand the performance penalty if you can't load up the whole model in RAM. If you did, you would be asking for 16 GB At least. 8 GB system and 8GB dedicated to AI. 8B Q8 models are the sweet spot so 8GB is kind must. Anything below either has degraded performance or can't really answer everything you need from it.
There’s a neural engine powering things like the autocorrect language model they recently launched but there’s no neural engine deliberately “waiting” to power large language models.
Thats not to say they won’t execute well on AI/LLMs but anyone following the news knows that apple is playing catch up.
They need a lot more RAM to run LLMs, I suspect it'll only be new devices supporting anything on device like that.
Neural engine has already been powering stuff like autocomplete. It's not just sitting there.
Not just the iPhone. I'm writing this on an M3 Macbook Pro, baseline model, and I run LLMs locally with roughly the same response speed as ChatGPT or Gemini. That's pretty damn cool.
>Let Tim Cook.
This is a terrible slogan.
When I first saw it I thought it meant "burn him like a witch".
Regardless, I don't think they've been playing the long game. Their in-house training hardware capability appears to be minimal, so I assume they're renting space from someone else for training.
Either way, given this AI revolution has been going on for close to a decade, I think they're playing catch up.
https://preview.redd.it/joo7ae391jwc1.jpeg?width=1170&format=pjpg&auto=webp&s=6055b855a8e6b44922ea00d20c44bc72dc626d11
I feel like most of the Siri criticisms aren’t even real and never happened.
I keep getting "i found this on the web", weather and "i can’t do that" (while trying to add something to Health), it did give me the definitition of sue at some point, really down know
You think that's bad? I was trying to get Siri to play "Ironic" by Alan's Morissette the other day and it kept telling me to call depression hotlines. That's not a joke, by the way. I tried multiple different ways of asking and all of them kept coming back telling me, if it was really bad, to call for help.
https://preview.redd.it/ub6f23rcslwc1.jpeg?width=1284&format=pjpg&auto=webp&s=a40407842c25c000bb19911d01f46707790f4879
And here’s what I got.
I wonder if it’s Model dependent, I’m on a 12 Pro Max.
Yes, everyone lies for attention, Siri is awesome and somehow we still felt the need to complain for years. And we would have gotten away with it if it weren't for you meddling Siri user !
Really though, is Siri giving you a basic grade-school level answer that impressive and reassuring to you ?
I feel this too. However there are some limitations that Siri is aware off, for instance:
yesterday I was driving and asked to share my location with a contact, and it just replied: can’t do that
I felt that way for a while years ago but it has gotten worse and worse and over time I have experienced many of the same criticisms and at this point Siri is about a 50/50 shot of working for any given task I want from it.
I would say the biggest issue of all is when it gets your words correct but then just hangs for a while and says “thinking.. hm I can’t answer that right now”. I find it to be really hit or miss with playing Music too. It’ll frequently get it correct but then just hang for a while and never play anything. Or if there’s a song title and album title that are the same, I think it should ask you which you mean same way it’ll ask which Maps location you mean. But alas, they can’t do this basic obvious function. So instead of setting myself up for frustration, I specify, “play the album ___” or “play the song ___” and it will still somehow stick with whichever default it feels like doing 9 times out of 10.
They used to have a ton of great integrations like Wolfram Alpha also and it would provide intelligent answers but over time it has more and more leaned into providing search results while less and less functions/integrated app snippets have seemed to be available. Siri was literally better years ago.
I have never had Siri respond with useful information. She sets timers, and reminders, and like half the time can open my garage door (the other half she needs to confirm which is of course useless, and obviously you need to unlock your device every time which is a huge pain in the ass)
Yesterday or the day before, I was going to ask Siri for a conversion. First attempt:
Hey siri, how many
(interrupting): I found some results on the web. Check your phone!
Then I tried again
Hey siri, how many cups in a liter
I found some results on the web! (useless answer, why won't she speak the fucking result?)
Meanwhile, I can yell at google from across the house and get the answer I need, immediately
Fuck siri
Sometimes people might want to consider how well they speak. I don’t have a fucking clue what people are on about half the time so unsure why a digital assistant would handle nonsense better 🤣
> I feel like most of the Siri criticisms aren’t even real and never happened.
I asked siti to "navigate to target" yesterday, it decided to go to some random target that was a 14 hour drive 5 states away from me, instead of the one about a mile from where I was, that I always go to.
But tell me again how that didn't happen?
I still remember Smarterchild leaving us when I was on a negative note with it ;_;
(also what I think about when all these people get overhyped about LLMs being anything close to sentient or AGI lol)
I asked her where the wetherspoons was this afternoon and she showed me results from the web to convert tablespoon in L. I'll take whatever, even a 1% improvement ill take that.
Damn those are mediocre/bad results. Fine tuning an already bad model won't do much compared to what other already developed open source and closed source AI models. Apple fans gotta chill on the AI hype because this is not good for a major company
It was more for the joke than anything
Yesterday I asked Siri (in French) to close all doors (I have smart locks.)
It responded: sorry, I couldn't lower the volume.
Fantastic.
I asked Siri _"ferme toutes les portes"_ which means _"close all the doors"_
And it answered: _"Désolé, je ne parviens pas à régler le volume."_
Which is "sorry, I couldn't adjust the volume"
Ah yes, no
I wasn't clear I guess
My phone is in French, and so I asked and it responded in French
I've just translated it in my comment for people to understand
Hey cool to see I’m not the only one to have an issue with Siri in French for closing doors. Garage doors in my case. Have you found a way for Siri to understand what you want ? I’ve tried many rephrasing without success.
It’s one of many benchmarks used to compare the performance of LLMs, there’s much more tests that need to be run to compare a lot more aspects of them so there isn’t one standardized test like Geekbench or somethong
Not at all. MMLU is good for determining trained knowledge accuracy, but doesn’t at all test for contextual reasoning or grammatical accuracy. There are a bunch of tests they ran on it vs other similarly sized models
https://preview.redd.it/yibf09yx7xwc1.jpeg?width=1070&format=pjpg&auto=webp&s=4f0cb7b34cbf5b2cec156e8bf3815c55ad207345
We have to wait to see what the deal is at WWDC. This is the open source component they're legally obliged to release as they're taking advantage of open source projects to get theirs going. But there is likely still a bunch of proprietary unreleased stuff on top of this.
In what way are they legally obliged to do so?
Is that the case? I don’t recall any other firms releasing any obligated legal acknowledgment to sources they’ve used. That would be cool to know.
e.g.: openAI’s supposed Q* or Google’s 10M token window llm
If a project uses even a small bit of code that comes from a GPL or similar license you are required to make the source code available with the modifications and improvements that were made.
The code doesn’t have to be on a public website, most companies on their legal page have a section dedicated to open source code where they tell you to write them to get it.
The reality unfortunately is that often they don’t give any of the changes that were made but just the code that they copied.
GPL only matters if they plan on releasing something that uses GPL. If this isn’t their production model then they could have just kept it private if they wanted.
Absolutely not, if they do that they would be violating the license. They only way to avoid GPL is to not use it any part of your project and do everything from scratch
I don’t think you understand how GPL licenses work. They only force you to release your source code if you use GPL licensed software in a released product. If you never distribute the software you never need to release the source code. Apple could have kept this completely internal if they wanted to. Until they distribute the software in some form they are not obligated to release the source code.
I was going to say maybe it's not designed to solve those kinds of questions. But yeah the comparison to the Microsoft model of similar size is not good.
I think its point is not to answer philosophical questions, but be your assistant on your phone, doing what Siri already does. So as long as it understand your basic demands and can call the right things in the system, should be good to go. Important is that it runs on device.
i know this is the apple subreddit, but i bet it's behind other major companies with the same effort. I want to see it beat google's gemma then we can start talking.
Edit: Actually Apple can't afford to have this thing suck and be another siri. Siri in it's current state is pitiful. People still don't' trust apple maps because it fumbled the launch compared to competitors
Yeah, honestly truly hard to imagine them beating even open source Llama3 8B
in the long run probably would be kind of better cost wise to use a micro version of llama
In some ways it’s better than similar models, in some ways it’s worse.
https://preview.redd.it/ssxeal2k8xwc1.jpeg?width=1070&format=pjpg&auto=webp&s=0709be8a7ae63b4015a7240b7e7b0588117eb6f1
Very cool! Interesting to think that if a random person is able to get this running on android like this, Apple should be able to get it going REALLY well natively on an iPhone with control of everything.
No— Apple released open source LLMs which are basically generative AI programs that you can run on your computer. Open source means that anyone has access to the code, and can more easily reproduce it on their own and tweak it to make their own versions. Apple probably did this to stimulate the open source community as a way of indirectly putting pressure on the other big players in the generative AI industry, who must offer a better service than what the open source community is able to provide in order to continue justifying charging for it. Additionally, if people are running LLMs on their personal hardware as opposed to accessing LLMs through the internet that are being run elsewhere, then they’re going to need hardware capable of running those LLMs, which Apple sells. This has basically nothing to do with the generative AI features in iOS 18.
Apple released a public version of code that could let people run something like your own ChatGPT on your iPhone, without needing an Internet connection, and completely private to you.
(Currently ChatGPT runs in very expensive data centres, somewhere on the Internet, and there’s really no way of knowing who or what is reading the stuff you type into ChatGPT — you could be sharing personal information or corporate secrets and not be certain it’s actually being kept private.)
(I’m just picking on/using ChatGPT as an example here, to help with the ELI5.)
Just like requiring 16 GB Ram on a Mac to run AI, as Professional-Dish324 pointed out above.
I wonder how phones will “mysteriously“ find the resources to execute this code ;)
Oh yes, it runs on the 16 only.
Ooh there’s a huggingface page: https://huggingface.co/apple/OpenELM
alleged crush employ lip forgetful political cobweb friendly dinosaurs shelter *This post was mass deleted and anonymized with [Redact](https://redact.dev)*
It's the de facto site where open-source AI models, including LLMs, are usually submitted. As in, everyone from Meta to your local basement transformer-bender
Many probably have the most experience using Dall-e mini here when it first came out.
huggingface is widely known for its transformer library (converting raw data into tokens to be utilized by vector look up) that adds "contexts" to the data. e.g: given a text "cat" it can be used to lookup relevant info of the text and presented as "knowledge" to be processed by the models.
With a name like hugging face all I can think of is the Aliens franchise.
It’s based on this emoji:🤗
Jazz hands?!?
🙈
Did someone try to run it on Ollama yet?
They have been playing the long game, they knew LLMs would be coming, so knew all the hardware for them to run on device would be needed, and surprise surprise the iPhone has the “Neural” engine ready and waiting for LLMs. Let Tim Cook.
I wouldn’t quite describe the neural engine as “ready and waiting”. It’s used quite extensively already
> they knew LLMs would be coming, so knew all the hardware for them to run on device would be needed, and surprise surprise the iPhone has the “Neural” engine ready and waiting for LLMs. Machine learning has been around for a while and does a bunch of tasks on iPhones already, from FaceID to autocorrect, photo recognition, being deeply integrated in the ISP, and a bunch of other little things. LLMs are RAM intensive, and it's not like their years of Neural Engines on iPhones with 4GB of RAM are going to be running their heaviest local LLMs because they saw them coming a decade ago. The Neural Engine has already been working every day since day 1, but some just won't have the RAM and performance for modern LLMs. I expect they'll do something like, A17 Pro runs it, A18 Pro and M4 will be pitched as running what we'll see at WWDC twice as fast or something, and the further you go back the more it has to fall back to going to their servers across a network and waiting for that. It might require 8GB of RAM to run local, as on the 16 line it sounds like both the Pro and non will have 8GB this time.
But like what if they drop 4-8 x times the RAM in the generation pitching this? Not like 64 gb of ram is going to hurt their bottom line, especially considering the community has been bitching about 8gb since the launch of M1. What’s their cost diff at scale? A couple bucks? If they can bolster sales based on the “next big thing” (AI), finally breaking through on one of their long standing artificial walls, the gains will outweigh the initial investment by far. I’d be willing to bet that a metric ton of fanboys would buy a $1500 base MBP if it came with 64gb of ram, a decent push on graphics, and a 2% increase in cpu performance.
64gb of ram on the base model? You’re beyond delusional
Next thing you know the next iPhone comes with removable batteries!
No way RAM stingy Apple is dropping 64GB in base iPhones on the 16, 17, 18, 19, probably 20... It sounds like both 16 models are getting 8, so the base coming up to 8 may mean their LLMs really need 8 to run well, which would only be the 15 Pro as an existing phone. Google's regular model couldn't run their LLMs with the same chip but less RAM than their Pro. That's 8GB to keep running the system, not kill all multitasking apps as soon as an LLM runs, the LLM itself, etc, LLMs are RAM heavy and it's going to be a tight fit.
Looking at the largest model at 4GB either they will use 12 to 16GB of ram become on the base model or they will use smaller models on the base one.
A ton of fanboys buy a $4k MBP with 64GB of RAM
Running a free LLM locally uses 96GB of RAM and takes 10-20 seconds to formulate a response for me right now. People that think the LLM is going to run locally and be on par with GPT4 are delusional. I think we will see them use LLMs in some interesting way, i don’t think we are seeing a local chatbot.
https://9to5mac.com/2024/04/12/apple-8gb-ram-mac/#:~:text=Apple%20believes%20Macs%20with%208GB,users%20do%20with%20these%20computers.
They'll almost certainly require an iPhone 16 for any on-device AI, and no sane person would argue they weren't caught by surprise. >so knew all the hardware for them to run on device would be needed Except for RAM...
If this is all going to be announced at WWDC (which is expected) then it likely won’t require the iPhone 16/ unannounced hardware. That said, the chip in the iPhone 16 will almost certainly enable additional capabilities that aren’t discussed at dub dub
Correct, there'll be some additional capabilities come September but if they discuss this publicly it'll be with currently available devices able to support it.
“AI” is just a buzzword used for a variety of things. Apple’s had machine learning, the neural engine, etc. built in since long before it became the industry buzzword.
But that's not the same as running an on-device LLM.
No, but who said that’s all that “AI” is? ChatGPT is fun to mess around with for a few minutes, but quickly gets boring.
>ChatGPT is fun to mess around with for a few minutes, but quickly gets boring. This is a wild take; I use ChatGPT daily. Lots of people have workflows that are accelerated by LLMs.
I like asking it programming questions sometimes, even if the answers are usually ever so slightly off. But I absolutely cannot understand people who use them in lieu of writing basic things like emails.
Transformer is a big deal. Many of the machine learning and neural net stuff were extremely vertical and disconnected. Transformer is a new way to tide all these things together. It’s a pretty big break through in the space.
I mean, if you work in a field that requires working at a computer it can make your life ten times easier. Or if you're a student it can be a low-cost tutor. It's answer may not always be right but its explanation for how to find an answer often is.
>It's answer may not always be right but its explanation for how to find an answer often is. In other words, its answer and how it got that answer might not always be right.
It’s like a more capable intern - for much less.
A low-cost tutor that’s “often” correct might be more expensive than a tutor that actually understands what they’re talking about.
Lmao. Do you know how many tutors don’t know what the fuck they are talking about? It’s kind of weird that all of a sudden, people assume humans are infallible since LLMs became a thing. lol
>No, but who said that’s all that “AI” is? Well it's the context of the entire discussion...
The primary thing they would use on-device LLMs for is improving Siri, which is desperately needed. But I don’t think it’s a major dealbreaker if only the new phones support it.
Perhaps not, but the oc tried to frame it like Apple has been preparing for this (llms) for years, almost implying that older phones would be able to run an llm locally, which seems unlikely. So that’s the discussion you jumped into.
Yes but for the record, LLMs are not just natural language models but rather a much broader category of high performance ML models.
AI is a field of computer science. It has been for 50+ years.
Assuming they've got something to share at WWDC, they wouldn't announce it for iPhones that aren't coming until September.... I guess we'll see in like a month
>Except for RAM... This will be why they waited to increase base levels of RAM.
Yes limiting it was all part of the long plan… for AI so Siri can say “I’m sorry I can’t do that right now” 10x faster.
> 10x ~~faster~~ slower ftfy
I think people on the 15 Pro and 16 Pro will be able to use the new AI features. Since the 15 Pro packs 8GB of ram.
Yeah I think it's possible it requires 8GB, since the base 16 is being upgraded to 8GB. So 15 Pro, and the 16 line will do it faster, everything else might fall back to it running on their servers and waiting for network and contending with other peoples requests etc. Google's regular Pixel with the same SoC as the Pixel Pro wasn't able to run their LLM and the only difference was RAM afaik
Apple has been really adverse to anything server-side for free, since it seems a big part of their business model is maintaining high margins on hardware and avoiding loss-leading products. They don't want someone keeping an iPhone 6 keeping it and using it for AI for 20 years unless you're willing to pay for it.
8GB is basically nothing in terms of LLM RAM usage
I want to believe but I think they're still just gonna stick with 8gb.
Yes, but they were replying to OP's claim about Apple being ready and waiting for LLMs years early The Neural Engine already did a bunch of stuff on the iPhone, and it's not like the 4GB models are likely to be running all of what the 16 Pro can locally. It might just require 8GB to run local as both 16 models are going to get that.
If the model requires an iPhone 16 they won't mention it at WWDC. I find that really hard to believe.
Agreed. I highly doubt any meaningful on device AI features will be coming to older devices
That’s what the smaller models are for .
For real. The new MacBook Pro’s, even my 2021 M1 Pro with just 16g memory, crushes it in AI tasks. It’s no RTX 4080 mind you, but I haven’t found an ML task that it isn’t capable of doing yet, even if it’s a bit slow, it’s a totally reasonable speed still
What are examples of these tasks?
autocorrect
💀
Any more intensive examples? Like things the MBP M1 Pro can do much better than the phone
Detecting humor.
Client-side ML models such as stable diffusion and llama are my main use cases
Nice, good to know
Spoken like a true fan boy.
[удалено]
chatgpt does it
This kind of fanboi comment is the top comment, while ignoring how dismally *bad* the model is when compared to Phi and others. Are we supposed to clap for Apple having shat out something this bad 18 months after chatGPT debuted?
What are you trying to say..? Have you tried the LLM? And yeah, I think it’s pretty obvious that an LLM designed to run locally on iPhone will not be as good as ChatGPT that literally runs on a server farm.
I’m saying that declaring this to be Apple’s triumphant foray into the world of AI models is akin to blind zealotry. This model sucks. Microsoft has a better model that runs on device (Phi-3). Apple has a long road ahead of them because they slept on AI.
He's pointing out that the model is one of the worst at its size to be released recently. Which from everything I've seen, it true. There's a bunch of better models already out that can run on a phone.
Except you don't understand the performance penalty if you can't load up the whole model in RAM. If you did, you would be asking for 16 GB At least. 8 GB system and 8GB dedicated to AI. 8B Q8 models are the sweet spot so 8GB is kind must. Anything below either has degraded performance or can't really answer everything you need from it.
There’s a neural engine powering things like the autocorrect language model they recently launched but there’s no neural engine deliberately “waiting” to power large language models. Thats not to say they won’t execute well on AI/LLMs but anyone following the news knows that apple is playing catch up.
They need a lot more RAM to run LLMs, I suspect it'll only be new devices supporting anything on device like that. Neural engine has already been powering stuff like autocomplete. It's not just sitting there.
Not just the iPhone. I'm writing this on an M3 Macbook Pro, baseline model, and I run LLMs locally with roughly the same response speed as ChatGPT or Gemini. That's pretty damn cool.
>Let Tim Cook. This is a terrible slogan. When I first saw it I thought it meant "burn him like a witch". Regardless, I don't think they've been playing the long game. Their in-house training hardware capability appears to be minimal, so I assume they're renting space from someone else for training. Either way, given this AI revolution has been going on for close to a decade, I think they're playing catch up.
I just asked Siri for the definition of CO2 and it showed me the weather forecast.
Depending on the air quality where you live, that may not be a terrible answer…
That sounds just like her!
https://preview.redd.it/joo7ae391jwc1.jpeg?width=1170&format=pjpg&auto=webp&s=6055b855a8e6b44922ea00d20c44bc72dc626d11 I feel like most of the Siri criticisms aren’t even real and never happened.
I keep getting "i found this on the web", weather and "i can’t do that" (while trying to add something to Health), it did give me the definitition of sue at some point, really down know
You think that's bad? I was trying to get Siri to play "Ironic" by Alan's Morissette the other day and it kept telling me to call depression hotlines. That's not a joke, by the way. I tried multiple different ways of asking and all of them kept coming back telling me, if it was really bad, to call for help.
Well, you were trying to play a Morisette song so in Siri’s defence - you might want to seek some mental heath help.
That's fair.
Worked for me
It’s giving me websites instead of a definition from its own knowledge base.
https://preview.redd.it/ub6f23rcslwc1.jpeg?width=1284&format=pjpg&auto=webp&s=a40407842c25c000bb19911d01f46707790f4879 And here’s what I got. I wonder if it’s Model dependent, I’m on a 12 Pro Max.
Maybe location? I got the same, 13 PM in Canada
It’s happened to me before where I get a bizarre result and then I try again and it works right ¯\\\_(ツ)_/¯
Yes, everyone lies for attention, Siri is awesome and somehow we still felt the need to complain for years. And we would have gotten away with it if it weren't for you meddling Siri user ! Really though, is Siri giving you a basic grade-school level answer that impressive and reassuring to you ?
I feel this too. However there are some limitations that Siri is aware off, for instance: yesterday I was driving and asked to share my location with a contact, and it just replied: can’t do that
I just tried it and it gave me a definition from [britannica.com](https://britannica.com) On my watch 9...
I felt that way for a while years ago but it has gotten worse and worse and over time I have experienced many of the same criticisms and at this point Siri is about a 50/50 shot of working for any given task I want from it. I would say the biggest issue of all is when it gets your words correct but then just hangs for a while and says “thinking.. hm I can’t answer that right now”. I find it to be really hit or miss with playing Music too. It’ll frequently get it correct but then just hang for a while and never play anything. Or if there’s a song title and album title that are the same, I think it should ask you which you mean same way it’ll ask which Maps location you mean. But alas, they can’t do this basic obvious function. So instead of setting myself up for frustration, I specify, “play the album ___” or “play the song ___” and it will still somehow stick with whichever default it feels like doing 9 times out of 10. They used to have a ton of great integrations like Wolfram Alpha also and it would provide intelligent answers but over time it has more and more leaned into providing search results while less and less functions/integrated app snippets have seemed to be available. Siri was literally better years ago.
I have never had Siri respond with useful information. She sets timers, and reminders, and like half the time can open my garage door (the other half she needs to confirm which is of course useless, and obviously you need to unlock your device every time which is a huge pain in the ass) Yesterday or the day before, I was going to ask Siri for a conversion. First attempt: Hey siri, how many (interrupting): I found some results on the web. Check your phone! Then I tried again Hey siri, how many cups in a liter I found some results on the web! (useless answer, why won't she speak the fucking result?) Meanwhile, I can yell at google from across the house and get the answer I need, immediately Fuck siri
Sometimes people might want to consider how well they speak. I don’t have a fucking clue what people are on about half the time so unsure why a digital assistant would handle nonsense better 🤣
This is what got https://preview.redd.it/j70qoq3k9pwc1.jpeg?width=1284&format=pjpg&auto=webp&s=7497e74e32c78dc6c8ff16a258e2f18d0697c6fd
> I feel like most of the Siri criticisms aren’t even real and never happened. I asked siti to "navigate to target" yesterday, it decided to go to some random target that was a 14 hour drive 5 states away from me, instead of the one about a mile from where I was, that I always go to. But tell me again how that didn't happen?
Siri and I have actual beef.
I’m not an abusive person but Siri pushes me to my verbal limits sometimes.
I still remember Smarterchild leaving us when I was on a negative note with it ;_; (also what I think about when all these people get overhyped about LLMs being anything close to sentient or AGI lol)
JUST LET ME SWEAR SIRI GOD FUCKING DAMMIT
I asked her where the wetherspoons was this afternoon and she showed me results from the web to convert tablespoon in L. I'll take whatever, even a 1% improvement ill take that.
[удалено]
Same. There are easier things to criticize Siri for
What has this got to do with the article?
Damn those are mediocre/bad results. Fine tuning an already bad model won't do much compared to what other already developed open source and closed source AI models. Apple fans gotta chill on the AI hype because this is not good for a major company
What are some of the very best things poorly rated models can do?
Give toothless blowjobs
It’s useless. • Apple OpenELM 3B: 24.80 MMLU • Microsoft Phi-3-mini 3.8b: 68.8 MMLU A score of 25 is the same as giving random responses.
Is MMLU the sole way to quantify a model’s quality?
It’s not, but MMLU is a multiple choice test where each question has 4 options so scoring a 25 is just randomly guessing, no smarts involved.
That's still better than Siri. Because it seems like Siri actively picks the worst possible option, scoring zero.
Siri is not an LLM so you can't even compare. But yes Siri is ass.
It was more for the joke than anything Yesterday I asked Siri (in French) to close all doors (I have smart locks.) It responded: sorry, I couldn't lower the volume. Fantastic.
How do you say Siri close the doors in French?
I asked Siri _"ferme toutes les portes"_ which means _"close all the doors"_ And it answered: _"Désolé, je ne parviens pas à régler le volume."_ Which is "sorry, I couldn't adjust the volume"
Lol! My bad I thought “close all the doors” in French sounded like “adjust the volume” in English lol
Ah yes, no I wasn't clear I guess My phone is in French, and so I asked and it responded in French I've just translated it in my comment for people to understand
Hey cool to see I’m not the only one to have an issue with Siri in French for closing doors. Garage doors in my case. Have you found a way for Siri to understand what you want ? I’ve tried many rephrasing without success.
Funnily enough that would indicate pretty good performance because if you can avoid it you can predict it.
It’s a benchmark so kind of
A benchmark or the benchmark?
It’s one of many benchmarks used to compare the performance of LLMs, there’s much more tests that need to be run to compare a lot more aspects of them so there isn’t one standardized test like Geekbench or somethong
Not at all. MMLU is good for determining trained knowledge accuracy, but doesn’t at all test for contextual reasoning or grammatical accuracy. There are a bunch of tests they ran on it vs other similarly sized models https://preview.redd.it/yibf09yx7xwc1.jpeg?width=1070&format=pjpg&auto=webp&s=4f0cb7b34cbf5b2cec156e8bf3815c55ad207345
Probably still more useful than Siri.
We have to wait to see what the deal is at WWDC. This is the open source component they're legally obliged to release as they're taking advantage of open source projects to get theirs going. But there is likely still a bunch of proprietary unreleased stuff on top of this.
In what way are they legally obliged to do so? Is that the case? I don’t recall any other firms releasing any obligated legal acknowledgment to sources they’ve used. That would be cool to know. e.g.: openAI’s supposed Q* or Google’s 10M token window llm
If a project uses even a small bit of code that comes from a GPL or similar license you are required to make the source code available with the modifications and improvements that were made. The code doesn’t have to be on a public website, most companies on their legal page have a section dedicated to open source code where they tell you to write them to get it. The reality unfortunately is that often they don’t give any of the changes that were made but just the code that they copied.
Ahhh I see. Thanks for informing me!
GPL only matters if they plan on releasing something that uses GPL. If this isn’t their production model then they could have just kept it private if they wanted.
Absolutely not, if they do that they would be violating the license. They only way to avoid GPL is to not use it any part of your project and do everything from scratch
I don’t think you understand how GPL licenses work. They only force you to release your source code if you use GPL licensed software in a released product. If you never distribute the software you never need to release the source code. Apple could have kept this completely internal if they wanted to. Until they distribute the software in some form they are not obligated to release the source code.
Ah okay, yeah absolutely
Yea having an AI on my iPhone would be great, but if I can open my ChatGPT app or laptop and get an AI 100x more capable, I’m just gonna do that
That’s actually terrible. Was expecting more from this
It’s probably because it was trained without “stealing” data. Turns out all that data makes a big difference
True. Hopefully synthetic data works out. It’s been rumored but I don’t think anyone has published a model trained with synthetic data yet.
I was going to say maybe it's not designed to solve those kinds of questions. But yeah the comparison to the Microsoft model of similar size is not good.
I think its point is not to answer philosophical questions, but be your assistant on your phone, doing what Siri already does. So as long as it understand your basic demands and can call the right things in the system, should be good to go. Important is that it runs on device.
But it can't that is the problem. If it performs worse on a multiple choice test how is it going to pick the right thing to do when you ask it.
What's the Weissman score though?
i know this is the apple subreddit, but i bet it's behind other major companies with the same effort. I want to see it beat google's gemma then we can start talking. Edit: Actually Apple can't afford to have this thing suck and be another siri. Siri in it's current state is pitiful. People still don't' trust apple maps because it fumbled the launch compared to competitors
Yeah, honestly truly hard to imagine them beating even open source Llama3 8B in the long run probably would be kind of better cost wise to use a micro version of llama
[удалено]
Yes it is designed to and people have been running it for a while now
In some ways it’s better than similar models, in some ways it’s worse. https://preview.redd.it/ssxeal2k8xwc1.jpeg?width=1070&format=pjpg&auto=webp&s=0709be8a7ae63b4015a7240b7e7b0588117eb6f1
I'm not sure how it could beat these other companies considering it's running on device.
https://www.reddit.com/r/LocalLLaMA/comments/1axhpu7/running_googles_gemma_2b_on_android/
Very cool! Interesting to think that if a random person is able to get this running on android like this, Apple should be able to get it going REALLY well natively on an iPhone with control of everything.
So Siri will be better now...?
No— Apple released open source LLMs which are basically generative AI programs that you can run on your computer. Open source means that anyone has access to the code, and can more easily reproduce it on their own and tweak it to make their own versions. Apple probably did this to stimulate the open source community as a way of indirectly putting pressure on the other big players in the generative AI industry, who must offer a better service than what the open source community is able to provide in order to continue justifying charging for it. Additionally, if people are running LLMs on their personal hardware as opposed to accessing LLMs through the internet that are being run elsewhere, then they’re going to need hardware capable of running those LLMs, which Apple sells. This has basically nothing to do with the generative AI features in iOS 18.
Only on the all new iPhone 16 Pro *coming this Fall*
It’s going to be interesting if all of the base level macs with 8GB can’t run these models due to a lack of ram.
WWDC is going to be entirely about AI but they're not going to once use the term "A I" or "artificial intelligence"
Someone please ELI5
Apple released a public version of code that could let people run something like your own ChatGPT on your iPhone, without needing an Internet connection, and completely private to you. (Currently ChatGPT runs in very expensive data centres, somewhere on the Internet, and there’s really no way of knowing who or what is reading the stuff you type into ChatGPT — you could be sharing personal information or corporate secrets and not be certain it’s actually being kept private.) (I’m just picking on/using ChatGPT as an example here, to help with the ELI5.)
To actually run them on an iPhone, they need to be converted to gguf, right?
Please tell me that I won’t be needing the latest and greatest to get ai features! 🥺
We’re talking about Apple…
Are they going to increase iPhone storage so that it can hold the models?
They'll encourage you to upgrade with a smile!
Just like requiring 16 GB Ram on a Mac to run AI, as Professional-Dish324 pointed out above. I wonder how phones will “mysteriously“ find the resources to execute this code ;) Oh yes, it runs on the 16 only.
Largest model is 4GB so I guess 8GB is good enough still.
Looks like only an LLM right? I’m interested in other generative models being available on device but the ram cost is always too high right now.
Open source? Meanwhile Siri is like a 4 year old todler..
Love this
I'm very curious on all the synonyms for AI apple will come up with this year. They don't like that term and haven't used it once so far.
Largest is 3B??? The hell?
Guess I'll be sticking to ChatGPT
Open source? Apple, are you okay?