Sparks of AGI

Sparks of AGI

With the announcement of Open AI receiving funding from Microsoft, we’ve been treated to some very interesting papers from Microsoft studying the capabilities of Open AI’s GPT-4 – the neural network behind Chat GPT 4, the premium version. GPT-4 is a multi-modal (i.e. text and images) Machine Learning (ML) model that shows some amazing and even unexpected abilities.

In their famous (infamous?) paper “Sparks of Artificial General Intelligence: Early experiments with GPT-4” (found here) they explore that GPT-4 abilities and limits.

Here is a fascinating experiment taken from the paper (page 55) to see if GPT-4 could ‘understand’ theory of mind:

A scenario prompt written out as code as a question posed to Chat GPT-4, see paragraph after image for textual breakdown.

So, Alice has put a file in a folder in Dropbox, and Bob moves it without telling Alice. Alices tries to find the file. A human can tell immediately that Alice is going to look in the wrong spot because she doesn’t know Bob moved it, so in her mind the image is still in the old folder. This may seem like a simple example to a human because we’re so naturally good at ‘theory of mind’. We can put ourselves in the place of Alice and realize Alice can’t know that Bob moved it.

Artificial Intelligence (AI) has had an enormously hard time trying to deal with what to humans seems like such a simple question. Even AIs specifically built to try to handle challenges like this rarely can pass such tests. As you can see, GPT-4 was able to pass this test! And what is even more interesting is that it passed this test without ever having been trained to!

In fact, the Microsoft researchers note that GPT-4 can act like other kinds of ‘narrow AI’ without having been trained to do it. For example, Microsoft trained a specialized machine learning (ML) model to find Personally Identifiable Information (PII) so that it can be scrubbed from a dataset for legal reasons. Some kinds of PII are obvious, like a name- but other kinds are less obvious. For example: is an IP address personal information? For some people it is a single value that personally identifies them, so it should count as personal information even if it isn’t so obvious. Worse yet, sometimes individual fields are not personal information but when taken together they are. So, catching personal information is a difficult problem even for a human. Microsoft trained an ML model, called Presidio, to help find PII in a dataset. When they had GPT-4 compete with Presidio the winner was GPT-4 even though it wasn’t trained for this task!

Or put another way, GPT-4 seems to be able to mimic other kinds of specially trained ML models without additional training. This ‘general intelligence’ is something new with Large Language Models (LLMs) like GPT-4. And it is an exciting discovery. (see p. 69 of the paper)

If you are interested in learning more about Microsoft’s paper either download their 155-page paper; or I offer a shorter primer on my podcast that is aimed at the study of Artificial General Intelligence (AGI). See The Theory of Anything Podcast, episode 70: Sparks of Artificial Intelligence?

Also, Mindfire and our team of talented and experienced software engineers are always looking to tackle new and challenging projects, including ones with AI/ML elements. If you have an idea or would be interested in working with us, be sure to reach out via our contact-us page: CONTACT-US


comments powered by Disqus

Follow Us

Latest Posts

subscribe to our newsletter