ChatGPT, a new AI conversation model made publicly available byOpenAI, has taken the Internet by storm and become the subject of countless memes using the eerily accurate responses the software can produce.OpenAIis a non-profit that those familiar with the artificial intelligence space will know well. The company was founded in San Francisco by Sam Altman (current CEO), Ilya Sutskever (current Chief Scientist), and other engineers - with additional financial assistance from Elon Musk. While Musk is noted to be a co-founder of the company, and remains a donor, he resigned from the board in 2018.
The founders raised over $1 billion total, which has since been spent developing various AI technologies such as theimage generator DALLE-2among other notable computational feats. ChatGPT is a reworking of the company’s core language model (GPT-3.5) to engage in conversations and answer questions from a human partner. It is an extension of the Instruct version of the GPT model whose focus was education, which may explain ChatGPT’s propensity for answering human questions.
RELATED:Games To Play If You Like AI: The Somnium Files
Understanding The Technology Behind OpenAI’s GPT
In order to understand ChatGPT it is crucial to understand what OpenAI’s GPT, or Generative Pre-trained Transformer, actually is. GPT is a language model, which is a system used by computational linguists in order to define the probability of any given sequence of words.Using these probabilities, language modelscan effectively predict, parse, and even produce sentences and sequences of words, creating the impression that the computer “understands” the language it models.
Thoroughlyexplaining the field of computational linguisticsalone could fill hundreds of books, but the most important thing to understand is that GPT is a kind of language model which studies a large dataset, and then calculates the probability of any given sequence of words. GPT is specifically an autoregressive language model, meaning it uses previous information in its dataset to predict upcoming information.
Beyond being a language model,OpenAI’s concept of a Generative Pre-trained Transformerinvolves training the language model via a two-stage process. The first step is called “pre-training,” which involves GPT being left to learn from the dataset based on preset parameters. The second step involves fine-tuning GPT’s continued learning with human supervision to optimize results for targeted tasks. GPT is technically called a general-purpose learner, meaning its learning is not focused on completing singular tasks. This mixture of freedom and human oversight is critical to ensuring a flexible and accurate language model.
The State of OpenAI’s GPT Language Model
GPT’s original release in 2018 did not make abig public splash the way that GPT-2 and 3have. GPT-2, essentially the same technology as the original GPT but with a significantly larger dataset, was the last instance of the software that was truly open-source. GPT-2’s open-source nature has made it popular among enthusiasts, and it has even found its way into game projects like the AI text-adventure generatorAI Dungeon.AI Dungeonand many other users of the GPT language models have made the switch to the superior GPT-3 and 3.5, which were a move away from OpenAI’s so-called open approach to its technology.
Microsoft essentially purchased the exclusive rights to GPT-3in 2020, after making a $1 billion investment in OpenAI in 2019. Individuals and companies can currently only lease access to the technology’s API, meaning Microsoft can use the model to generate output but no one can modify the actual code behind the model.
This monopolistic control oversuch an influential piece of modern AI researchhas raised many alarm bells, but the continued success of DALLE-2, ChatGPT, and other commercial applications of the GPT model have been difficult to deny. As deep-learning algorithms and AI technology advance, a return toOpenAI’s open-source principles as these tools mature would promote greater safety and accessibility.
