Quick notes on Language Models

A study submitted in July 2023 reported that GPT is getting “dumber”, with decreased performance on math calculus and visual reasoning. This caused a bit fuzz on the internet, but anyway, I’d like to make some notes here. An important note about theses notes (jokes and puns aside): they all come from head and from my experience and world knowledge. I did not consult any reference to make them (except for the links, of course), so be aware of this. Here are the notes:

1. GPTs and Language Models based on Transformers learn from text data (as you may know) and represent this acquired knowledge as ontologies (roughly speaking, a network of interconnected concepts). Thus, prompting a GPT or any other language model is basically “querying an ontology”, of course, in a very smart and practical way. Hence, a Language Model actually does not do calculus per se, but queries the “ontology” to try to figure the result. Hence (again), we shouldn’t except that much of GPT in the “math side of the force”.
2. Despite that, we can expect from GPT and Language Models that they be good inference machines, namely, machines that generate new knowledges based on what they have and what they are queried for. They could be considered a “smart search engine”.
3. For those who don’t know, search engines were also a big fuzz in the past just like Language Models are being today. In this sense, we just need to learn how to use them effectively and with good sense as we did with search engines.
4. I’ve recently discovered this other Language Model service that is a good concurrent to ChatGPT and Bard, and the best of all: you don’t need to Login to use it. For me, that’s really awesome. This service also provides a client for LLaMa, the “Meta’s GPT”, in which you also don’t need to login.