“Roughly speaking, they take huge amounts of data, search for patterns in it and become increasingly proficient at generating statistically probable outputs — such as seemingly humanlike language and thought.”
Noam Chomsky, Ian Roberts, and Noam Chomsky: The False Promise of ChatGPT” The quote above is from the second paragraph. It confirms my earlier descriptions of how ChatGPT and others work. The article goes on to explain a few central ideas about how human thought works.
“The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; it seeks not to infer brute correlations among data points but to create explanations.”
I love the well-known quip from Humboldt that the human mind makes “infinite use of finite means”.1
The article points out the amazing feat that virtually every human accomplishes within a few years from birth – the acquisition of language – really the development of speech and grammar from the environment surrounding them.
The center of the article is here:
“The crux of machine learning is description and prediction; it does not posit any causal mechanisms or physical laws. Of course, any human-style explanation is not necessarily correct; we are fallible. But this is part of what it means to think: To be right, it must be possible to be wrong. Intelligence consists not only of creative conjectures but also of creative criticism. Human-style thought is based on possible explanations and error correction, a process that gradually limits what possibilities can be rationally considered. (As Sherlock Holmes said to Dr. Watson, “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.”)
But ChatGPT and similar programs are, by design, unlimited in what they can “learn” (which is to say, memorize); they are incapable of distinguishing the possible from the impossible.
……..
Whereas humans are limited in the kinds of explanations we can rationally conjecture, machine learning systems can learn both that the earth is flat and that the earth is round. They trade merely in probabilities that change over time.
For this reason, the predictions of machine learning systems will always be superficial and dubious.”
The article includes the required exchange between one of the authors and ChatGPT. “Would it be moral to terraform Mars?”
The article closes:
“In short, ChatGPT and its brethren are constitutionally unable to balance creativity with constraint. They either overgenerate (producing both truths and falsehoods, endorsing ethical and unethical decisions alike) or undergenerate (exhibiting noncommitment to any decisions and indifference to consequences). Given the amorality, faux science and linguistic incompetence of these systems, we can only laugh or cry at their popularity.”
My closing thought is that if software engineers cannot discipline themselves, we must step in to regulate their experimentation. We have all sorts of laws and regulations about how physicians and others perform experiments on humans. Seems that a similar logic should be applied here. What would have come of our attention economy that is sucking up enormous time and energy from billions of people if we had some regulations for experimentation in place? In the present situation, the human fallout from the attention economy is just an “external cost”2 to the capitalist enterprises engineering our attention to feed their profits. This is no different than the pollution that we are so familiar with. Another external cost for capitalist enterprises.
Footnotes
- Humboldt: https://en.wikipedia.org/wiki/Wilhelm_von_Humboldt
- Here is a concrete example of external costs defined within the railway industry: “External costs are costs generated by transport users and not paid by them but by the society as a whole such as congestion, air pollution, climate change, accidents, noise but also up- and down-stream processes, costs for nature and landscape or additional costs in urban areas.” – https://uic.org/support-activities/economics/article/external-costs
