PromptLayer is a comprehensive platform for prompt engineering, management, versioning, testing, evaluations, and observability. Their mission is to enable teams—especially those with non-technical subject matter experts—to rigorously iterate on AI prompts and track their performance.

"We really believe the best prompt engineers are not ML engineers," says Jared Zoneraich, co-founder of PromptLayer. "They're the subject matter experts, the lawyers, doctors, therapists, content writers, etc. We're really trying to enable them and build UX from the ground up to make AI useful for them."

The challenge

PromptLayer needed to provide their users with flexible capabilities to transform and manipulate data within AI workflows. While they had built 20 different predefined transformation types, customers kept requesting custom functionality that couldn't be covered by these standard options:

The engineering team, consisting of about 10 engineers, faced the challenge of supporting these custom use cases without constantly adding new node types or compromising on the ease of use for non-technical users.

Solution: Riza-powered code execution

PromptLayer integrated Riza to provide a flexible code execution capability within two key product areas:

1. Evals: PromptLayer's evaluation system allows users to run batch tests on AI models, transform outputs, and measure performance. Riza powers the Code Execution column type, which users commonly use to define custom scoring functions.
2. Agents: PromptLayer's AI agent builder enables users to create chains of operations that include prompts and data transformations. Riza powers the Code Execution node type, which lets users define any desired logic via code, including fetching data from a remote source or transforming data from a previous step.

Code execution is now a common tool people reach for—earning it a top spot in the UI. "We put the Code Execution option as one of the first ones," says Zoneraich. "Sometimes writing code is just easier than figuring out the PromptLayer-specific way to do something. And non-engineers can even use ChatGPT to write the code snippet they need."

Why Riza?

When evaluating solutions, PromptLayer needed:

• Fast execution. "Riza was just so fast. I didn't want to spin up a sandbox for every single row when processing thousands of inputs," explains Zoneraich. PromptLayer's evaluation workflows often need to process batch runs of 10,000+ rows of data, with each execution running on separate Redis workers. The performance requirements made traditional sandboxing solutions impractical.
• Easy implementation. "The first version took less than 30 minutes to integrate. It was super easy," recalls Zoneraich. With PromptLayer's small engineering team handling a product with extensive surface area, they needed a solution that wouldn't require significant integration work or maintenance overhead.
• Reduced engineering burden. "If we had to build it ourselves, we probably wouldn't have prioritized it. It probably wouldn't be done today," says Zoneraich. As a startup, PromptLayer prefers to focus on their core product rather than building and maintaining secure code execution infrastructure. Riza allowed them to offer this capability without diverting engineering resources.

Riza was just so fast. I didn't want to spin up a sandbox for every single row when processing thousands of inputs.

Jared Zoneraich, co-founder of PromptLayer

Results

In the first three months since implementing Riza:

• Over 400,000 code execution steps have been run in the Eval product.
• 650,000 code execution calls have been made in the Agents product.
• 1 in 3 PromptLayer "power users" (those with 3+ evaluations) are using the Riza-powered code execution feature.

Benefits

The integration has delivered unexpected benefits as well:

1. Fewer feature requests: "It decreases engineering burden on our team because it enables our customers to just find new things and build new stuff," says Zoneraich. Instead of constantly building custom transformation nodes or workflow steps, PromptLayer can now point users to the code execution capability. This has dramatically reduced the backlog of feature requests while giving users the tools to solve their unique challenges.
2. Support for edge cases: "It's probably kept us a lot of customers. In dev tools, if you hit 80% of a user's needs, they're going to leave your tool," Zoneraich emphasizes. By offering Riza's code execution, PromptLayer can support the critical 20% of edge cases without having to anticipate every use case in advance.
3. Unlocked new workflows: Users discovered they could use the code execution feature to make network requests and implement callbacks in agents. This capability allowed customers to build custom integrations in PromptLayer, such as triggering notifications or updating external systems at the end of an agent workflow. The flexibility of code execution gave users this power out of the box.

Impact on PromptLayer's business

PromptLayer's integration with Riza has helped them serve enterprise customers with mission-critical AI applications.

"We have teams using these products for end-user applications that are generating many millions in revenue per year, and PromptLayer is how they keep these AI products from regressing," says Zoneraich.

More importantly, it's helped them deliver on their core vision: "What I'm really excited about is that we've discovered this paradigm where the engineering team sets up the application, and the domain expert is in the driver's seat for editing and testing prompts. With the help of Riza and our rigorous eval system, engineering is actually able to reliably let non-technical users edit problems and test them themselves."

About Riza

Riza provides a secure, high-performance code execution environment that you can integrate easily via APIs. Riza offers full control over the execution environment, including fine-grained network access control, and optimized performance for fast response times.