There is a book—Output, published by The MIT Press, and written by Lillian-Yvonne Bertram and Nick Montfort. The book is an anthology of seven decades of English-language outputs from computer generation systems, introducing us to the fascinating history of machine-written texts by the computers.
Nick Montfort wrote an essay on The MIT Press sharing interesting examples from the book—to help us understand how computers manipulate language to generate text.
To quote Nick—”Alison Knowles, a poet and Fluxus artist, collaborated with composer and programmer James Tenney, who held a computer programming workshop in the New York apartment of Knowles and her husband Dick Higgins. Tenney worked with Knowles’s poem structure, her word lists, and the FORTRAN IV programming language.“
Knowles designed a model based on “A house of plastic / in a metropolis / using natural light / inhabited by people from all walks of life.”
Here is how it is processed.
The post shows that the modern LLMs and ChatGPT are not doing anything really innovative—generating text from the available information (call it content, language, knowledge center) was always an application of computers use, for different reasons.
Joseph Weizenbaum, an MIT computer scientist developed ELIZA in the year 1966—the first program that used Natural Language Programming (NLP). ELIZA could identify the input keywords and it respond with a pre-programmed answer based on certain login and the model itself. This work served as the foundation of development of small and later on complex LLMs.
So, LLMs are not new.
PS: I am not a historian or researcher into the history of computer generated text. The essay by Nick caught my attention and as a technology practitioner, I thought it is a good idea to have some historical context into text generation by the computers.
