You could perhaps make the argument that it’s a statistical token predictor, but that is about as useful as boiling down various weather services as “just statistics” or the economy as “just statistics”.
Making a language model that speaks a language is not that hard, but the world of science underlying how this is done is anything but simple. Saying it’s just statistics is ridiculously reductive like saying your response to this comment is just chemistry. Context driven tokenization, byte level byte pair encoding, RoBERTa, fine tuning methods, direct preference optimization, dataset curation and management, and curriculum learning for targeted performance and memory are things that are being developed and refined very fast (like weekly or monthly breakthroughs sometimes) and with pretty staggering performance increases. It still is not for everyone because power tools injure, but instead of saying “just a statistics engine” say what you really mean “I don’t understand it, but I believe XXX is a bad use case for LLMs.”
To a lesser extent that any company is “programming AI”. Not in the way that you mean it; curriculum learning, guard rails, and fine running are all extremely indirect. Nobody had their hands specifically in a model’s parameter space directly.
Aight, ima bite. LLMs are just statistical word predictors.
I would agree that parts of weather services are weather pattern predictors (which likely do include some machine learning nowadays) and certain aspects of economics are statistic predictions (the rest is either monopoly or pseudoscience lol), so yeah both are statistics.
I’ll also agree that the statistics of “AI” is difficult and honestly beautiful. Transformers really are ingenious and the methods for optimizing them are as well.
However, I don’t think it is comparable to “your response to this is just chemistry”
There’s a significant difference between these systems.
One is a machine emerging from physics over billions of years to eventually comprehend various aspects of the universe and social interaction as a byproduct, experiencing the illusion of free will, and deciding to respond/not-resopond to a comment.
The other, is a machine built with the sole purpose of predicting the “right” next word. It’s a novel area of science, but it is a science and it is still just a word (or “token” if you want to be pedantic) predictor.
Saying your response to the comment is just chemistry is more like saying AI is just the laws of electromagnetism; both of which are stupid. But saying a specific statistical modeling method is in fact a statistical modeling method is just a fact.
And I’m not saying this as someone who is pure “anti-AI” since it does have its uses. The work going into LLMs is incredible and the breakthroughs for LLMs typically can be extended to other machine learning problems.
If you can train a model to notice unsafe code, who do you think will find bugs/exploits easier in a large code base? A model you can parallelize that never gets tired or a pen-tester/programmer who will eventually need to sleep? Obviously the model wins for vulnerabilities that are common and that is significant because if they are common vulnerabilities, you probably have them in your code base.
Machine translation is, of course, the standard perfect use case for LLMs. With newer models even idioms can get translated well, and there are many more uses outside the realm of text-based models too.
But again, all this is still 100% statistics. These models are statistical models. They are using previous data and specified parameters, “guardrails”, etc. to build a model which when given some new data will give an output with probabilistic accuracy. That is literally the core of statistics.
On the note of “no one has their hands in a model’s parameter space directly” that’s actually false and there’s been more active study into finding and “inhibiting” certain parameters that have correlated with “lies.”
But yes, the point of AI (like that of any statistical model) is to remove the need for manual tuning. So yeah the point is mostly to not mess with parameters directly, but to build a framework by which you can basically just pump training data in and get a model which makes high accuracy predictions. That’s the same point for polynomial regression, there are just more applications for neural nets than simple regression models.
Sure there are some who are trying to build true artificial intelligence by replicating the neuroanatomy of living things, which could be non algorithmic and possibly even eventually sentient. But LLMs? Yeah no, those are just a fancy new statistical modeling method. It’s new so there’s more to explore and lots of stuff that hasn’t been tried yet, but that’s just how new things are in science and math.
I took some time to reflect on this post because I don’t prefer to be the moron, hopefully that didn’t slow you down because your response is a good response.
To start I think that hyping up the thinking people do as a counter argument is exactly the same thing I was doing by describing recent advancements in AI and I’m not sure it moves us foward, so I will count that as a wash.
And you basically hit it for me: my biggest gripe is that we are not doing one word at a time predictors any more. So functionally yes it’s true, but practically we are doing next word prediction for LLMs in the same way you’re doing next word prediction when you talk: you know where you’re going and we build sentences one word at a time.
I also don’t think it’s fair to say that “oh fundamentally LLM stuff is just electromagnetism” because while that is literally true, I think it’s still a better analogy to say statistics vs chemistry. The comments are going to make you feel a certain way, you’ll make a response based on that feeling and what you know, and we will continue. I think that’s pretty functionally the same as “here’s an open prompt, we need to answer it, let’s take a statistical approach to the specific wording we will use”.
Now that we are including end goals, reflection, and rule sets, I don’t think it’s fair to say it’s just one word at a time prediction anymore because training and optimization are happening at the prompt and response scale, not the word scale anymore.
I have not read anything recently that is purporting to do something useful by editing NN weights directly, but that doesn’t mean it isn’t happening. I think that is actually just a side discussion in the end.
In the end, if there is evidence of planning and then adding words to meet this plan, even if it goes one word at a time, I think we are escaping “statistical word generator”. And if you say that we didn’t escape that threshold, I would suggest that when we talk we are doing the same thing: we understand grammar at a pretty fundamental level, but when it comes to vocab there are only a handful of words that make sense and we are making that decision in a way that is not altogether different from LLM sentence generation. I think the only sane way to disprove that is if you want to go looking for substantially offbeat phrasing or expression that would be outside the bounds of “statistical regularity” that LLMs are using.
Unless you want to show a history of totally whackadoodle phrasing for ideas in lemmy, LLMs are no more staticstical word predictors than you are.
Okay, yes, we aren’t doing single words at a time (technically some models do chunk long words anyway so even from near the beginning we weren’t doing “singular” words at any time)
However, you are both right and wrong in your assertion that LLMs our equivalent human responses.
The translation of thoughts to words and words to motor functions both can be approximated by ANNs. And yes, the work done to select words we use is a probabilistic process like you describe. We hear patterns in language and that makes us more likely to use that phrasing. The more you hear a phrase the more likely you are to use it over another and when two or more phrases would communicate what you want to say your brain basically just picks one.
So, If speech production (or sentence construction) was what you meant by saying “it’s the same for human responses,” then yes, we agree. Both are probabilistic word generators and likely work in similar ways. (In fact I think place cells were found in Wernicks area (?) or one of the other speech corteces which means some of our word selection is likely similar to the results from transformer architectures)
However, if you meant the entirety of human response—as in from hearing/reading a comment, thinking about it, responding—is the same as current LLMs generating text. I strongly disagree.
The actual process of “thinking” is not something an ANN (especially a non-recurrent one) can do. The ability to ruminate on thoughts and make changes/learn-new-things simply by trying to formulate ideas before even deciding to comment cannot be accomplished with a pre-trained static net, not even one with memory or the illusion of memory like current LLMs. (Not to mention that identity also plays a large role in our responses and it too cannot arise from current deterministic architectures)
As for me asserting human response is chemistry is more like asserting AI is electromagnetism, there are many reasons why, but the simplest illustration would be this:
I think it is entirely possible to build an inorganic but still functional human brain on electrical hardware. (In other words, full blown transhumanism or at the very least, “AGI”) If human response is chemistry in organics, it would be electromagnetism in silicon.
AI/llm is just a statistical word predictor.
You could perhaps make the argument that it’s a statistical token predictor, but that is about as useful as boiling down various weather services as “just statistics” or the economy as “just statistics”.
Making a language model that speaks a language is not that hard, but the world of science underlying how this is done is anything but simple. Saying it’s just statistics is ridiculously reductive like saying your response to this comment is just chemistry. Context driven tokenization, byte level byte pair encoding, RoBERTa, fine tuning methods, direct preference optimization, dataset curation and management, and curriculum learning for targeted performance and memory are things that are being developed and refined very fast (like weekly or monthly breakthroughs sometimes) and with pretty staggering performance increases. It still is not for everyone because power tools injure, but instead of saying “just a statistics engine” say what you really mean “I don’t understand it, but I believe XXX is a bad use case for LLMs.”
To a lesser extent that any company is “programming AI”. Not in the way that you mean it; curriculum learning, guard rails, and fine running are all extremely indirect. Nobody had their hands specifically in a model’s parameter space directly.
Aight, ima bite. LLMs are just statistical word predictors.
I would agree that parts of weather services are weather pattern predictors (which likely do include some machine learning nowadays) and certain aspects of economics are statistic predictions (the rest is either monopoly or pseudoscience lol), so yeah both are statistics.
I’ll also agree that the statistics of “AI” is difficult and honestly beautiful. Transformers really are ingenious and the methods for optimizing them are as well.
However, I don’t think it is comparable to “your response to this is just chemistry”
There’s a significant difference between these systems.
One is a machine emerging from physics over billions of years to eventually comprehend various aspects of the universe and social interaction as a byproduct, experiencing the illusion of free will, and deciding to respond/not-resopond to a comment.
The other, is a machine built with the sole purpose of predicting the “right” next word. It’s a novel area of science, but it is a science and it is still just a word (or “token” if you want to be pedantic) predictor.
Saying your response to the comment is just chemistry is more like saying AI is just the laws of electromagnetism; both of which are stupid. But saying a specific statistical modeling method is in fact a statistical modeling method is just a fact.
And I’m not saying this as someone who is pure “anti-AI” since it does have its uses. The work going into LLMs is incredible and the breakthroughs for LLMs typically can be extended to other machine learning problems.
If you can train a model to notice unsafe code, who do you think will find bugs/exploits easier in a large code base? A model you can parallelize that never gets tired or a pen-tester/programmer who will eventually need to sleep? Obviously the model wins for vulnerabilities that are common and that is significant because if they are common vulnerabilities, you probably have them in your code base.
Machine translation is, of course, the standard perfect use case for LLMs. With newer models even idioms can get translated well, and there are many more uses outside the realm of text-based models too.
But again, all this is still 100% statistics. These models are statistical models. They are using previous data and specified parameters, “guardrails”, etc. to build a model which when given some new data will give an output with probabilistic accuracy. That is literally the core of statistics.
On the note of “no one has their hands in a model’s parameter space directly” that’s actually false and there’s been more active study into finding and “inhibiting” certain parameters that have correlated with “lies.”
But yes, the point of AI (like that of any statistical model) is to remove the need for manual tuning. So yeah the point is mostly to not mess with parameters directly, but to build a framework by which you can basically just pump training data in and get a model which makes high accuracy predictions. That’s the same point for polynomial regression, there are just more applications for neural nets than simple regression models.
Sure there are some who are trying to build true artificial intelligence by replicating the neuroanatomy of living things, which could be non algorithmic and possibly even eventually sentient. But LLMs? Yeah no, those are just a fancy new statistical modeling method. It’s new so there’s more to explore and lots of stuff that hasn’t been tried yet, but that’s just how new things are in science and math.
LLMs are just statistical word predictors.
I took some time to reflect on this post because I don’t prefer to be the moron, hopefully that didn’t slow you down because your response is a good response.
To start I think that hyping up the thinking people do as a counter argument is exactly the same thing I was doing by describing recent advancements in AI and I’m not sure it moves us foward, so I will count that as a wash.
The next thing I will add here is an article I read… idk awhile ago but it shows the explainability of AI decision making in a more advanced way than like Shapley parameters for example: https://transformer-circuits.pub/2025/attribution-graphs/methods.html
And you basically hit it for me: my biggest gripe is that we are not doing one word at a time predictors any more. So functionally yes it’s true, but practically we are doing next word prediction for LLMs in the same way you’re doing next word prediction when you talk: you know where you’re going and we build sentences one word at a time.
I also don’t think it’s fair to say that “oh fundamentally LLM stuff is just electromagnetism” because while that is literally true, I think it’s still a better analogy to say statistics vs chemistry. The comments are going to make you feel a certain way, you’ll make a response based on that feeling and what you know, and we will continue. I think that’s pretty functionally the same as “here’s an open prompt, we need to answer it, let’s take a statistical approach to the specific wording we will use”.
Now that we are including end goals, reflection, and rule sets, I don’t think it’s fair to say it’s just one word at a time prediction anymore because training and optimization are happening at the prompt and response scale, not the word scale anymore.
I have not read anything recently that is purporting to do something useful by editing NN weights directly, but that doesn’t mean it isn’t happening. I think that is actually just a side discussion in the end.
In the end, if there is evidence of planning and then adding words to meet this plan, even if it goes one word at a time, I think we are escaping “statistical word generator”. And if you say that we didn’t escape that threshold, I would suggest that when we talk we are doing the same thing: we understand grammar at a pretty fundamental level, but when it comes to vocab there are only a handful of words that make sense and we are making that decision in a way that is not altogether different from LLM sentence generation. I think the only sane way to disprove that is if you want to go looking for substantially offbeat phrasing or expression that would be outside the bounds of “statistical regularity” that LLMs are using.
Unless you want to show a history of totally whackadoodle phrasing for ideas in lemmy, LLMs are no more staticstical word predictors than you are.
Okay, yes, we aren’t doing single words at a time (technically some models do chunk long words anyway so even from near the beginning we weren’t doing “singular” words at any time)
However, you are both right and wrong in your assertion that LLMs our equivalent human responses.
The translation of thoughts to words and words to motor functions both can be approximated by ANNs. And yes, the work done to select words we use is a probabilistic process like you describe. We hear patterns in language and that makes us more likely to use that phrasing. The more you hear a phrase the more likely you are to use it over another and when two or more phrases would communicate what you want to say your brain basically just picks one.
So, If speech production (or sentence construction) was what you meant by saying “it’s the same for human responses,” then yes, we agree. Both are probabilistic word generators and likely work in similar ways. (In fact I think place cells were found in Wernicks area (?) or one of the other speech corteces which means some of our word selection is likely similar to the results from transformer architectures)
However, if you meant the entirety of human response—as in from hearing/reading a comment, thinking about it, responding—is the same as current LLMs generating text. I strongly disagree.
The actual process of “thinking” is not something an ANN (especially a non-recurrent one) can do. The ability to ruminate on thoughts and make changes/learn-new-things simply by trying to formulate ideas before even deciding to comment cannot be accomplished with a pre-trained static net, not even one with memory or the illusion of memory like current LLMs. (Not to mention that identity also plays a large role in our responses and it too cannot arise from current deterministic architectures)
As for me asserting human response is chemistry is more like asserting AI is electromagnetism, there are many reasons why, but the simplest illustration would be this:
deleted by creator
You’re literally arguing with the thing they’re debunking by just saying the thing they’re debunking
deleted by creator