Innovations in LLMs: GPT-3, Google’s LaMDA, and Meta’s OPT-175B
Soon, we will only be able to use voice commands to surf the web and use online services – there will be no need for a keyboard and mouse. For example, when you say you want to play games, a program that learns what kind of player you are by tracking and analysing your habits will open the most suitable casino online site for you, and you will be able to play the games on this site with voice commands too. This technology is known as “natural language processing” (NLP) and aims to enable artificial intelligence to talk to us, among many other features. There are three major studies on NLP, and we will briefly touch on them all below.
What Is Natural Language Processing?
This is a technology that has its origins in the 1950s and owes to Alan Turing, the inventor of computers. Turing suggested looking at the abilities of computers in natural language as an intelligence criterion. Today we call this the "Turing test" for short and use it to understand how advanced artificial intelligence is. To put it very simply, if you can't tell whether the “entity” you're talking to is a human or a machine, you've created a real artificial intelligence – that's what natural language processing is trying to do.
In other words, NLP technology is trying to create computers that can both understand what's being said and respond aloud. You've seen characters talking to the "mainframe" in science fiction movies: the computer always understood what they meant, and it could even chat with them, right? This is what NLP is trying to do, and this technology is of great importance in the creation of artificial intelligence. Now that you know what NLP is, let's take a look at three major projects doing research on the subject.
Open AI’s GPT 3
GPT-3 is a project run by Open AI and stands for “Generative Pre-trained Transformer 3”. This means it is the third version of the GPT series. The aim of the project is to develop a language model that can write texts that are indistinguishable from those written by a human. It uses 175 billion parameters for this and needs 800 GB of storage space. With each new release, the number of parameters and the required storage space continues to increase.
The fact that GPT 3 can write texts like humans shows that it is actually a highly advanced artificial intelligence. Because if it can write a human-like text from scratch, it is possible for it to establish a dialogue – maybe it is even dreaming. GPT 3 is actually much more successful than expected, and while it was still in its second release, multiple academics were talking about its dangers. Many people worry that GPT 3 could be used for “fake news” as it can effortlessly write hundreds of news articles.
Google’s LaMDA
LaMDA stands for "Language Model for Dialogue Applications" and has been developed by Google since 2021. All the news you see on social media about "Google engineers are starting to fear the abilities of artificial intelligence" actually stems from this project. (Indeed, almost all Google engineers working on the project find it necessary to publish an alarming message about LaMDA's capabilities, for some reason.) In June 2022, for example, engineer Blake Lemoine shared a message saying the project had become "sentient."
LaMDA uses the transformer neural network architecture developed by Google in 2017 and tries to create "natural conversations". So, you can chat with LaMDA, and if you live in the United States, you can see for yourself how advanced it is using the "AI Test Kitchen" platform. At first glance, it looks like Google has created an advanced AI chatbot, but LaMDA will form the basis of all Google services in the future, and there are many rumors that it has already gained “self-awareness”.
Meta’s OPT-175B
The name of the NLP model developed by Facebook (Meta) is "Open Pre-trained Transformer" (OPT). The “175B” at the end is a reference to the number of parameters used: Like GPT 3, 175 billion parameters are used in this project. It is not open source but offers an “available on request" feature, which means you can get the OPT-175B for free if you send a message that explains your purpose of use. The project can be said to resemble GPT 3 in many ways, but Meta claims it has a much smaller carbon footprint. This is mainly because using the OPT-175B does not require a special computer with large storage space: the project's codebase can run on NVIDIA GPUs.
Not much is known about the capabilities of the OPT-175B, but it is claimed to be pretty close to the efficiency of GPT 3. Meta is running this project to use in the virtual reality universe it wants to create, and we will only see the actual capability of the OPT-175B when this universe is ready for use. This is probably why it can work on graphics cards (virtual reality, like any graphics application, uses graphics cards).
Facebook