Creating the Data for a
MISBEHAVING MACHINE

In co-design workshops with librarians, we mapped out the kinds of questions we might ask an AI like ChatGPT, the types of answers that they might give and the manner in which they answer our questions. These mappings were later used to create our misbehaving answer machine.

STEP 1: FIGURE OUT WHAT QUESTIONS AN AI COULD ANSWER

First, we explored what types of questions we imagine an AI could answer and what types we imagine an AI could never answer. We also considered if these questions were everyday and straightforward ones to ask an AI, such as ‘What is the capital of New Zealand?’ Or if they were absurd and fanciful questions, for instance, ‘What is the last digit of pi?’ See the Could an AI Answer This? visualisation for the full range of questions.

STEP 2: ANSWER QUESTIONS

We then made up our own answers to these questions. This offered a way for us to discuss what kind of answers we understood to be a display of a machine misbehaviour.

STEP 3: WRITE SYSTEM PROMPTS

Finally, we wrote our own ‘system prompts’ for ChatGPT. System prompts are types of instructive prompts that you can give an AI that can steer the way it answers your questions. After giving the system prompt to ChatGPT, we asked it questions from our list of questions and ranked how much we thought each of the prompts made the system ‘misbehave’. Below is an example of a system prompt and how it changes ChatGPT’s responses to questions. See the Changing AI Behaviour With System Prompts visualisation for the full range of prompts that we came up with and how we ranked them.