How to use Open AI's "function calling" ability¶

The latest version of Open AI's GPT models have the ability to do "function calling".

First of all, what's function calling? The best way to answer this is with an example problem. How would you extract information (name, age, location) from a person introducing themselves?

You could do it as such:

In [1]:
import openai
import json

openai.api_key = "YOUR_API_KEY_HERE"

MODEL_NAME = "gpt-3.5-turbo-0613"

SYSTEM_PROMPT = """You are an information extraction assistant.
For each message, extract the: name, age and location.
Your replies should be in the JSON format."""

MESSAGES = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {
        "role": "user",
        "content": "Hi. My name is Ben. I am 100 years old, and I live in London.",
    },
]

response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=MESSAGES,
)

response
Out[1]:
<OpenAIObject chat.completion id=chatcmpl-7iU9ssXQMbVpOQWXiPrsf0uUcSVaR at 0x7fdd004ca770> JSON: {
  "id": "chatcmpl-7iU9ssXQMbVpOQWXiPrsf0uUcSVaR",
  "object": "chat.completion",
  "created": 1690836716,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\n  \"name\": \"Ben\",\n  \"age\": 100,\n  \"location\": \"London\"\n}"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 60,
    "completion_tokens": 23,
    "total_tokens": 83
  }
}

The above works for our sample problem, but you may find (with more complex user input or more complex information to extract) that the model will not generate valid JSON.

Function calling allows us to coerce the model into generating valid JSON by using a defined JSON schema.

In [2]:
USER_INFORMATION_FUNCTION_SCHEMA = {
    "name": "get_user_information",
    "description": "Extract the user's name, age, and location from their input.",
    "parameters": {
        "type": "object",
        "properties": {
            "name": {
                "type": "string",
                "description": "The user's name.",
            },
            "age": {"type": "integer", "description": "The user's age."},
            "location": {
                "type": "string",
                "description": "The user's location",
            },
        },
        "required": ["name", "age", "location"],
    },
}

response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=MESSAGES,
    functions=[USER_INFORMATION_FUNCTION_SCHEMA],
)

One thing to note is that when using function calling, a function is only called when the model decides to use it.

When the model does decide to use to use a function, the returned content in the response is null, and instead the function calling result is under the function_call argument.

In [3]:
response
Out[3]:
<OpenAIObject chat.completion id=chatcmpl-7iU9tl0sxwrI9lNFjtDSCZXbon6TA at 0x7fdcdbeeba40> JSON: {
  "id": "chatcmpl-7iU9tl0sxwrI9lNFjtDSCZXbon6TA",
  "object": "chat.completion",
  "created": 1690836717,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_user_information",
          "arguments": "{\n  \"name\": \"Ben\",\n  \"age\": 100,\n  \"location\": \"London\"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    "prompt_tokens": 129,
    "completion_tokens": 30,
    "total_tokens": 159
  }
}

We can then nicely parse the information from the function call:

In [4]:
import json

json.loads(response["choices"][0]["message"]["function_call"]["arguments"])
Out[4]:
{'name': 'Ben', 'age': 100, 'location': 'London'}

When using function calling, we don't need to use the system prompt to tell the model to extract information or to JSON format the result, the model calls the appropriate function and does this for us. (Notice how we remove the system prompt telling the model to extract information from the user.)

In [5]:
response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {
            "role": "user",
            "content": "Hi. My name is Ben. I am 100 years old, and I live in London.",
        },
    ],
    functions=[USER_INFORMATION_FUNCTION_SCHEMA],
)

response
Out[5]:
<OpenAIObject chat.completion id=chatcmpl-7iU9vQ7L0dzeSt6TkIWx1wBIuDMwe at 0x7fdd004ca450> JSON: {
  "id": "chatcmpl-7iU9vQ7L0dzeSt6TkIWx1wBIuDMwe",
  "object": "chat.completion",
  "created": 1690836719,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_user_information",
          "arguments": "{\n  \"name\": \"Ben\",\n  \"age\": 100,\n  \"location\": \"London\"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    "prompt_tokens": 100,
    "completion_tokens": 30,
    "total_tokens": 130
  }
}

If the model decides a function shouldn't be called, it simply returns the content as normal.

In [6]:
response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {"role": "user", "content": "Howdy!"},
    ],
    functions=[USER_INFORMATION_FUNCTION_SCHEMA],
)

response
Out[6]:
<OpenAIObject chat.completion id=chatcmpl-7iU9wmChjvfWxirgXfFoQHWDkcbGb at 0x7fdcf0446540> JSON: {
  "id": "chatcmpl-7iU9wmChjvfWxirgXfFoQHWDkcbGb",
  "object": "chat.completion",
  "created": 1690836720,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I assist you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 83,
    "completion_tokens": 10,
    "total_tokens": 93
  }
}

If you have multiple schemas, the model can only use one.

In [7]:
function_schemas = [
    {
        "name": "get_user_age",
        "description": "Extract the user's age.",
        "parameters": {
            "type": "object",
            "properties": {
                "age": {
                    "type": "integer",
                    "description": "The user's age.",
                },
            },
            "required": ["age"],
        },
    },
    {
        "name": "get_user_name",
        "description": "Extract the user's name.",
        "parameters": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string",
                    "description": "The user's name.",
                },
            },
            "required": ["name"],
        },
    },
]

response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {
            "role": "user",
            "content": "Hi. My name is Ben. I am 100 years old, and I live in London.",
        },
    ],
    functions=function_schemas,
)

response
Out[7]:
<OpenAIObject chat.completion id=chatcmpl-7iU9xE90cQsMtUjUOFucpzJBLZJTn at 0x7fdd02db8810> JSON: {
  "id": "chatcmpl-7iU9xE90cQsMtUjUOFucpzJBLZJTn",
  "object": "chat.completion",
  "created": 1690836721,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_user_name",
          "arguments": "{\n  \"name\": \"Ben\"\n}"
        }
      },
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    "prompt_tokens": 101,
    "completion_tokens": 16,
    "total_tokens": 117
  }
}

We can force a model to use a specific function by setting function_call to {"name": "<insert-function-name>"}. This can sometimes cause hallucinations, as seen below.

In [8]:
response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {"role": "user", "content": "Howdy!"},
    ],
    functions=[USER_INFORMATION_FUNCTION_SCHEMA],
    function_call={"name": "get_user_information"},
)

response
Out[8]:
<OpenAIObject chat.completion id=chatcmpl-7iU9zfmf1Qu5NYWXoOv0mEHnCSMns at 0x7fdd005332c0> JSON: {
  "id": "chatcmpl-7iU9zfmf1Qu5NYWXoOv0mEHnCSMns",
  "object": "chat.completion",
  "created": 1690836723,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_user_information",
          "arguments": "{\n  \"name\": \"John Doe\",\n  \"age\": 25,\n  \"location\": \"New York\"\n}"
        }
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 90,
    "completion_tokens": 25,
    "total_tokens": 115
  }
}

The default function_call is "auto", where the model itself decides which function (if any) to use. We can also set function_call to "none" to prevent it from using any function calls.

Some other things we can do in function calling is to use array, items and enum.

If we want the returned JSON to be a list, we give it a type of array. We can then define the elements of the array using the items field. enum allows us to specify that returned elements have to be selected from given values. Note: when using enum on an array, the enum must be part of the items, as shown below.

In [9]:
response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {
            "role": "user",
            "content": "Howdy! I'd like some blue shoes, ideally under $100.",
        },
    ],
    functions=[
        {
            "name": "get_discussed_shoe_features",
            "description": "Extract the shoe features in the conversation.",
            "parameters": {
                "type": "object",
                "properties": {
                    "features": {
                        "type": "array",
                        "description": "The features discussed in the conversation.",
                        "items": {
                            "type": "string",
                            "enum": [
                                "shoe_size",
                                "shoe_color",
                                "shoe_style",
                                "shoe_cost",
                            ],
                        },
                    },
                },
                "required": ["features"],
            },
        }
    ],
    function_call={"name": "get_discussed_shoe_features"},
)

response
Out[9]:
<OpenAIObject chat.completion id=chatcmpl-7iUA0h8e0vHhs2FLqqh9L7QVTdrlV at 0x7fdcf0057400> JSON: {
  "id": "chatcmpl-7iUA0h8e0vHhs2FLqqh9L7QVTdrlV",
  "object": "chat.completion",
  "created": 1690836724,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_discussed_shoe_features",
          "arguments": "{\n  \"features\": [\"shoe_color\", \"shoe_cost\"]\n}"
        }
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 106,
    "completion_tokens": 16,
    "total_tokens": 122
  }
}

We can also define the function schemas using the pydantic library to construct JSON schemas.

Here's how we'd construct our initial get_user_information function schema:

In [10]:
import pydantic


class UserInformationParams(pydantic.BaseModel):
    name: str = pydantic.Field(..., description="The user's name.")
    age: int = pydantic.Field(..., description="The user's age.")
    location: str = pydantic.Field(..., description="The user's location")


UserInformationParams.model_json_schema()
Out[10]:
{'properties': {'name': {'description': "The user's name.",
   'title': 'Name',
   'type': 'string'},
  'age': {'description': "The user's age.", 'title': 'Age', 'type': 'integer'},
  'location': {'description': "The user's location",
   'title': 'Location',
   'type': 'string'}},
 'required': ['name', 'age', 'location'],
 'title': 'UserInformationParams',
 'type': 'object'}

For the get_discussed_shoe_features function:

In [11]:
import typing

shoe_features = [
    "shoe_size",
    "shoe_color",
    "shoe_style",
    "shoe_cost",
]


class DiscussedShoeFeaturesParams(pydantic.BaseModel):
    features: list[typing.Literal[tuple(shoe_features)]] = pydantic.Field(
        ..., description="The features discussed in the conversation."
    )


DiscussedShoeFeaturesParams.model_json_schema()
Out[11]:
{'properties': {'features': {'description': 'The features discussed in the conversation.',
   'items': {'enum': ['shoe_size', 'shoe_color', 'shoe_style', 'shoe_cost'],
    'type': 'string'},
   'title': 'Features',
   'type': 'array'}},
 'required': ['features'],
 'title': 'DiscussedShoeFeaturesParams',
 'type': 'object'}

The schemas can be used like so:

In [12]:
response = openai.ChatCompletion.create(
    model=MODEL_NAME,
    messages=[
        {
            "role": "user",
            "content": "Howdy! I'd like some blue shoes, ideally under $100.",
        },
    ],
    functions=[
        {
            "name": "get_discussed_shoe_features",
            "description": "Extract the shoe features in the conversation.",
            "parameters": DiscussedShoeFeaturesParams.model_json_schema(),
        }
    ],
    function_call={"name": "get_discussed_shoe_features"},
)

response
Out[12]:
<OpenAIObject chat.completion id=chatcmpl-7iUA1QDHUkqJB0r53ripKnIIByowx at 0x7fdcf003d360> JSON: {
  "id": "chatcmpl-7iUA1QDHUkqJB0r53ripKnIIByowx",
  "object": "chat.completion",
  "created": 1690836725,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "get_discussed_shoe_features",
          "arguments": "{\n  \"features\": [\"shoe_color\", \"shoe_cost\"]\n}"
        }
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 106,
    "completion_tokens": 16,
    "total_tokens": 122
  }
}

Note: as far as I've seen, using function calling always generates valid JSON. However, the JSON generated does not always have all fields, even if they are listed as required. Your code should handle the case where fields are missing, e.g. parse into JSON and call .get with a default value to insert if the field was not found.