Hey there! We're Riza and we make running untrusted code safe, easy, and even a little bit fun. We've found LLM tool use to go hand-in-hand with executing code.
The ReAct pattern is a technique where you equip an LLM with a set of "actions" (or functions) it can perform, and then have it alternate between "reasoning" and "acting" using one of those actions in an attempt to solve a problem.
Eighteen months ago Simon Willison wrote a short post implementing the ReAct pattern in Python. In that post, Simon equips OpenAI's gpt-3.5-turbo model with three tools:
- Wikipedia search
- search of Simon's blog
- a calculate() function which executes Python code generated by the LLM.
I recently found Simon's post again and it struck me how much progress we've seen. For instance, Simon implemented his own function calling using regular expressions, whereas today function calling is a native feature of many chat completion APIs including those from Anthropic and OpenAI. Simon used OpenAI's gpt-3.5-turbo, but we now have gpt-4o-mini which is far more capable and 60% cheaper.
Also, Simon included this disclaimer on his calculate() function:
calculate: - evaluate an expression using Python's eval() function (so dangerous! This should absolutely use something like a WebAssembly sandbox instead)
Today you can use Riza to execute code in a WebAssembly sandbox via a simple API call that looks like this:
riza = rizaio.Riza()
resp = riza.command.exec(language="PYTHON", code=code)
I thought it would be fun to reimplement Simon’s code using the API features and additional tools we have available today.
import json
import sys
import httpx
import openai
import rizaio
from pydantic import BaseModel
class Wikipedia(BaseModel):
query: str # Search query
def wikipedia(query):
print("searching wikipedia for: ", query)
return httpx.get(
"https://en.wikipedia.org/w/api.php",
params={
"action": "query",
"list": "search",
"srsearch": query,
"format": "json",
},
).json()["query"]["search"][0]["snippet"]
class Calculate(BaseModel):
code: str # Python code
def calculate(code):
print("calcuating: ", code)
resp = riza.command.exec(language="PYTHON", code=code, allow_http_hosts=["*"])
print(resp)
return resp.stdout
class SimonBlogSearch(BaseModel):
query: str # Search term
def simon_blog_search(query):
results = httpx.get(
"https://datasette.simonwillison.net/simonwillisonblog.json",
params={
"sql": """
select
blog_entry.title || ': ' || substr(html_strip_tags(blog_entry.body), 0, 1000) as text,
blog_entry.created
from
blog_entry join blog_entry_fts on blog_entry.rowid = blog_entry_fts.rowid
where
blog_entry_fts match escape_fts(:q)
order by
blog_entry_fts.rank
limit
1""".strip(),
"_shape": "array",
"q": query,
},
).json()
return results[0]["text"]
tools = [
openai.pydantic_function_tool(Wikipedia),
openai.pydantic_function_tool(SimonBlogSearch),
openai.pydantic_function_tool(Calculate),
]
client = openai.OpenAI() # Set OPENAI_API_KEY
riza = rizaio.Riza() # Set RIZA_API_KEY
sys_msg = 'Always look things up on Wikipedia if you have the opportunity to do so.'
messages = [
{ 'role': 'system', 'content': sys_msg},
{ 'role': 'user', 'content': sys.argv[1]},
]
completion = client.chat.completions.create(
model='gpt-4o-mini',
messages=messages,
tools=tools,
)
if completion.choices[0].message.tool_calls:
messages.append(completion.choices[0].message)
for tool_call in completion.choices[0].message.tool_calls:
args = json.loads(tool_call.function.arguments)
match tool_call.function.name:
case "SimonBlogSearch":
function_response = simon_blog_search(args["query"])
case "Wikipedia":
function_response = wikipedia(args["query"])
case "Calculate":
function_response = calculate(args["code"])
case _:
raise ValueError("unknown function")
messages.append(
{
"role": "tool",
"content": function_response,
"tool_call_id": tool_call.id,
}
)
completion = client.chat.completions.create(
model='gpt-4o-mini',
messages=messages,
tools=tools,
)
print(completion.choices[0].message.content)
Some examples
I took the original three queries and made sure they returned similar answers to Simon's original results.
What does England share borders with?
% python react.py "What does England share borders with?"
England shares its borders with Scotland to the north and Wales to the west.
Additionally, it is surrounded by the North Sea to the east, the English Channel
to the south, and the Atlantic Ocean to the west.
The previous answer was the shorter "England shares borders with Wales and Scotland.".
- Has Simon been to Madagascar? *
% python react.py "Has Simon been to Madagascar?"
Yes, Simon has mentioned the Pirates Museum in Antananarivo, Madagascar, in his
blog.
The previous answer was "Yes, Simon has been to Madagascar and visited the Pirates Museum located in Antananarivo". The new answer is a bit worse as it includes "in his blog".
Fifteen times twenty five
% python react.py "Fifteen * twenty five"
Fifteen multiplied by twenty-five equals 375.
The previous answer was "Fifteen times twenty five equals 375.".