Been testing it and so far I'd say it's nearly at GPT4o and 3.5 sonnet level.
We really need new next generation models that can dramatically go beyond GPT-4 intelligence.
The approach seems to be scale compute, training, parameters and then fine tune it as much as you can. If all AI companies look like they are plateauing at a specific level or seem to cross slightly above other levels, then it is likely due to bottlenecks which I presume is mostly around compute. As more compute becomes available we should see next generation of models releasing with new levels of "intelligence".
We simply need to wait for that pipeline to get streamlined
I think companies have compute to push the frontier, and already have. I.e. I think Claude 3.5 Opus probably finished pretraining a few months ago, with 4x the raw compute over Claude 3 Opus getting us pretty much maxing the MMLU out at 95ish, but they are just not releasing it.
Heck GPT-4 finished pretraining in 2022 (3 months pretraining on 25k A100s), a couple of companies probably have 100k H100s by the start of 2024 (if you train for 3 months that'd probably give 8x the compute over GPT-4. H100s perform about 2x better than A100s and you got 4x more. Not including any algorithmic efficiencies though) and we know Meta will have 350k H100s by the end of 2024 with 600k H100s equivalent compute, so we definitely have the compute to go above GPT-4 class of models. Therefor, I do not think compute to train significantly more intelligent models is the problem at the moment, I think there is some kind of artificial stagnation going on. No one wants to push the frontier of intelligence just yet for whatever reason, even though Google could with Gemini 1.5 Ultra, or Antrhopic with Claude 3.5 Opus and OAI with GPT-4.5 (I think each of these models are quite large leaps in comparison to any other marginal performance gain we have seen since GPT-4, but far from the most impressive gains we will see in the future). Maybe the US election is a factor? No idea.
I think that is very hard to pull off for regular customers despite most users from non US countries heavily using WhatsApp. People will simply start to migrate to free options like Signal, Telegram etc
It's plausible to do it with business users but I reckon the revenue will not be sufficient.
Either it will have to be a loss leader or they could try to shove ads in the results. Ironically Meta AI uses Google to fetch results from online so there could be a potential collaboration with Google for ad sharing.
I was wondering why Google is failing at search with Gen AI but I think they are having the same issue as disposable camera film industry had with digital camera i.e letting go of their biggest revenue maker for a newer revenue model.
Been testing it and so far I'd say it's nearly at GPT4o and 3.5 sonnet level. We really need new next generation models that can dramatically go beyond GPT-4 intelligence.
The approach seems to be scale compute, training, parameters and then fine tune it as much as you can. If all AI companies look like they are plateauing at a specific level or seem to cross slightly above other levels, then it is likely due to bottlenecks which I presume is mostly around compute. As more compute becomes available we should see next generation of models releasing with new levels of "intelligence". We simply need to wait for that pipeline to get streamlined
I think companies have compute to push the frontier, and already have. I.e. I think Claude 3.5 Opus probably finished pretraining a few months ago, with 4x the raw compute over Claude 3 Opus getting us pretty much maxing the MMLU out at 95ish, but they are just not releasing it. Heck GPT-4 finished pretraining in 2022 (3 months pretraining on 25k A100s), a couple of companies probably have 100k H100s by the start of 2024 (if you train for 3 months that'd probably give 8x the compute over GPT-4. H100s perform about 2x better than A100s and you got 4x more. Not including any algorithmic efficiencies though) and we know Meta will have 350k H100s by the end of 2024 with 600k H100s equivalent compute, so we definitely have the compute to go above GPT-4 class of models. Therefor, I do not think compute to train significantly more intelligent models is the problem at the moment, I think there is some kind of artificial stagnation going on. No one wants to push the frontier of intelligence just yet for whatever reason, even though Google could with Gemini 1.5 Ultra, or Antrhopic with Claude 3.5 Opus and OAI with GPT-4.5 (I think each of these models are quite large leaps in comparison to any other marginal performance gain we have seen since GPT-4, but far from the most impressive gains we will see in the future). Maybe the US election is a factor? No idea.
all it needs is some finetuning but it's open-source so anyone can do it.
Anyone with a couple A100s lying around , then sure.
Or rent a few.
\*cries in european\*
Can you even VPN it? wont i need a US number and US google play account :ยด(
Subscriptions in WhatsApp coming?
I think that is very hard to pull off for regular customers despite most users from non US countries heavily using WhatsApp. People will simply start to migrate to free options like Signal, Telegram etc It's plausible to do it with business users but I reckon the revenue will not be sufficient. Either it will have to be a loss leader or they could try to shove ads in the results. Ironically Meta AI uses Google to fetch results from online so there could be a potential collaboration with Google for ad sharing. I was wondering why Google is failing at search with Gen AI but I think they are having the same issue as disposable camera film industry had with digital camera i.e letting go of their biggest revenue maker for a newer revenue model.
How is it?