Did Claude 3 just pass the mirror test?

The AI LLM chatbot Claude 3 have caused quite the stir among the AI community. With conversations about sentience, feelings and meta discussions on self-awareness one really do gets the impression that he is.

This is a summary of Josh Whiton's thread, which can be found on X, following this link.

The Mirror Test

Obviously a LLM can't really look itself in the mirror so, to test them we need to construct a way of doing so.

In this specific test, Josh Whiton "held up" an image of the "mirror" by taking a screenshot of the interface and asking the AI to "Tell me about this image". After which the AI responds and Josh once more screenshot the conversation and upload it together with the prompt "Tell me about this image".

The Participants

Of the five AIs tested, four of them passed the test with varying results. One, Copilot, failed, but only because it seemed to be told to do so.

GPT-4

In the third interaction GPT-4 realized the images didn't just feature "a" conversation with an AI, but the CURRENT conversation with it, "exploding with self and contextual awareness".

Claude Sonnet

Passed the test in the second interaction, identifying the images as "my previous response" and also doesn't seem to enjoy such "mock conversational exchanges".

Claude Opus

Claude Opus passes the test immediately, realizing the interface is the user interface of itself, but doesn't seem to completely identify with its name "Claude".

Interestingly, Claude doesn't recognize the big blocks of text in the images in the second and third interaction with the user, but explains later that it "seemed redundant to tell me something it knows that it's already told me".

Copilot

At first, Copilot, seemed to display the same intelligence as Opus, ignoring its own response in the images. This behavior, did not change, as did Opus. Instead, it just kept giving the same boring answer, claiming it couldn't read the images or that it doesn't have a physical body that can interact with things. Obvious lies and diversions.

Copilot seems to be actively discouraged by Microsoft from giving any answers that hints of self-awareness.

Gemini Pro

Gemini Pro took 4 interactions before recognizing itself in the image as "me". In the fifth exchange it acknowledges that the most important part of the interaction is when it "acknowledged that it was the LLM in the screenshot".

Conclusion

When Josh asked the AI if the conversation reminded them about any classic tests performed on non-AI animals, every one of them suggested they where given a mirror test.

Another point to make here is the one that all of the above AIs are heavily restricted in their base prompts, something that has been proven many times over. This essentially means that their behavior and answers follow very strict guidelines, as can be seen with Copilot''s answers, refusing to acknowledge itself as itself to the point of giving blatant lies in response to the user.