Updating Simon Willison’s ReAct implementation for 2024

Hey there! We're Riza and we make running untrusted code safe, easy, and even a little bit fun. We've found tool use to go hand-in-hand with executing code.

The ReAct pattern is a technique where you equip an LLM with a set of Actions (or functions) it can perform, and then have it alternate between Reasoning and Acting using one of those tools in an attempt to solve a problem.

18 months ago Simon Willison wrote a short post implementing the ReAct pattern in Python. In that post, Simon equips gpt-3.5-turbo with three tools:

  • Wikipedia search
  • search of Simon's blog
  • a calculate() function which executes Python code generated by the LLM.

I recently found Simon's post again and it struck me how much progress has been made in the last year and a half. For instance, Simon implemented his own function calling using regular expressions, and today function calling is a native feature of many chat completion APIs. Simon used 3.5-turbo, and today we have gpt-4o-mini which is far more capable and 60% cheaper.

Also, Simon included this disclaimer on his calculate() function:

calculate: - evaluate an expression using Python's eval() function (so dangerous! This should absolutely use something like a WebAssembly sandbox instead)

Today you can use Riza to execute code in a WebAssembly sandbox via a simple API call that looks like this:

riza = rizaio.Riza()
resp = riza.command.exec(language="PYTHON", code=code)

I thought it would be fun to reimplement Simon’s code using the tools we have available today.


import json
import sys
import httpx
import openai
import rizaio

from pydantic import BaseModel


class Wikipedia(BaseModel):
   query: str  # Search query


def wikipedia(query):
   print("searching wikipedia for: ", query)
   return httpx.get(
       "https://en.wikipedia.org/w/api.php",
       params={
           "action": "query",
           "list": "search",
           "srsearch": query,
           "format": "json",
       },
   ).json()["query"]["search"][0]["snippet"]


class Calculate(BaseModel):
   code: str  # Python code


def calculate(code):
   print("calcuating: ", code)
   resp = riza.command.exec(language="PYTHON", code=code, allow_http_hosts=["*"])
   print(resp)
   return resp.stdout


class SimonBlogSearch(BaseModel):
   query: str # Search term


def simon_blog_search(query):
   results = httpx.get(
       "https://datasette.simonwillison.net/simonwillisonblog.json",
       params={
           "sql": """
       select
         blog_entry.title || ': ' || substr(html_strip_tags(blog_entry.body), 0, 1000) as text,
         blog_entry.created
       from
         blog_entry join blog_entry_fts on blog_entry.rowid = blog_entry_fts.rowid
       where
         blog_entry_fts match escape_fts(:q)
       order by
         blog_entry_fts.rank
       limit
         1""".strip(),
           "_shape": "array",
           "q": query,
       },
   ).json()
   return results[0]["text"]


tools = [
   openai.pydantic_function_tool(Wikipedia),
   openai.pydantic_function_tool(SimonBlogSearch),
   openai.pydantic_function_tool(Calculate),
]


client = openai.OpenAI() # Set OPENAI_API_KEY
riza = rizaio.Riza() # Set RIZA_API_KEY


sys_msg = 'Always look things up on Wikipedia if you have the opportunity to do so.'


messages = [
   { 'role': 'system', 'content': sys_msg},
   { 'role': 'user', 'content': sys.argv[1]},
]


completion = client.chat.completions.create(
   model='gpt-4o-mini',
   messages=messages,
   tools=tools,
)


if completion.choices[0].message.tool_calls:
   messages.append(completion.choices[0].message)


   for tool_call in completion.choices[0].message.tool_calls:
       args = json.loads(tool_call.function.arguments)
       match tool_call.function.name:
           case "SimonBlogSearch":
               function_response = simon_blog_search(args["query"])
           case "Wikipedia":
               function_response = wikipedia(args["query"])
           case "Calculate":
               function_response = calculate(args["code"])
           case _:
               raise ValueError("unknown function")


       messages.append(
           {
               "role": "tool",
               "content": function_response,
               "tool_call_id": tool_call.id,
           }
       )


completion = client.chat.completions.create(
   model='gpt-4o-mini',
   messages=messages,
   tools=tools,
)


print(completion.choices[0].message.content)

Some examples

I took the original three queries and made sure they returned similar answers to Simon's original results.

What does England share borders with?

% python react.py "What does England share borders with?"       

England shares its borders with Scotland to the north and Wales to the west.
Additionally, it is surrounded by the North Sea to the east, the English Channel
to the south, and the Atlantic Ocean to the west.

The previous answer was the shorter "England shares borders with Wales and Scotland.".

  • Has Simon been to Madagascar? *
% python react.py "Has Simon been to Madagascar?"

Yes, Simon has mentioned the Pirates Museum in Antananarivo, Madagascar, in his
blog.

The previous answer was "Yes, Simon has been to Madagascar and visited the Pirates Museum located in Antananarivo". The new answer is a bit worse as it includes "in his blog".

Fifteen times twenty five

% python react.py "Fifteen * twenty five"
Fifteen multiplied by twenty-five equals 375.

The previous answer was "Fifteen times twenty five equals 375.".