Quora is a Q&A platform where users can ask, answer, and peruse questions on a variety of topics. With 400 million monthly unique visitors, it’s an invaluable contributor to the world’s knowledge-sharing. Quora uses Modal Sandboxes to securely execute LLM-generated code in Poe, their AI chatbot platform. The team has saved 15% of ongoing engineering time by using Modal over maintaining an in-house solution.

Hello, Poe

In 2023, Quora launched Poe, an AI chatbot platform where anyone can deploy a public chatbot. With millions of monthly active users, Poe is the default destination for many AI builders to experiment with different models. Quora has since raised $75M to keep expanding Poe.

A code interpreter for Poe

Many of the LLM bots in Poe can generate code, and users expected to run that code in Poe rather than copy-pasting it to their editors. The Quora team needed a way to safely execute code in Poe in a completely isolated way, keeping that code separate from both the main Quora infrastructure and any other user’s session.

In-chat Python execution in a Poe chatbot

There were three key requirements for this feature.

1. Security, since LLM-generated code can’t be trusted by default.
2. Low latency, since chatbot responses need to be fast in order to feel conversational.
3. Reliability, since the product has millions of users and is expected to be polished.

To build this themselves, the team would have needed to design a container product that could maintain strict separation of execution environments. This would have required sophisticated systems engineering know-how, since sandbox building blocks like gVisor operate at the application kernel level to intercept system calls. Such an investment would have taken months of time upfront to enumerate and address every potential edge case—not to mention the continuous investment required to patch new risks.

Modal’s Sandbox product offered strict security guarantees without the need for Quora to operate low-level virtualization tech. Quora was already familiar with Modal and, due to Modal’s superior reliability over alternatives, had it recommended as the default deployment solution for users publishing their own Poe bots. This gave Quora the confidence to expand their usage into Modal Sandboxes.

A Modal Function by any other name

Modal Sandboxes are really just our core primitive—Modal Functions—minus our client running inside of them. This means that Quora got a battle-tested and continuously improving product right out of the box.

✅ Modal’s custom container stack, which we have invested years into making robust and secure, is already built on gVisor for enterprise-grade container isolation.

✅ Fast scalability is built in. Quora stress-tested Sandbox creation throughput to 1000 Sandboxes per second with no issue, allowing them to support thousands of users who might be generating code at any given point in time.

✅ Existing features like Tunnels come for free, too, allowing Quora to directly communicate with Sandbox containers from their servers for full customizability and control.

This is just the beginning. The Poe team is working on an under-the-wraps new product that will also leverage Modal Sandboxes for code execution. We’re looking forward to sharing more once they launch!

Build fast like Quora

Interested in running LLM-generated code safely? Get started today with Modal Sandboxes.

1. Install Modal: pip install modal
2. Create an account: python -m modal setup
3. Run:

import modal
app = modal.App.lookup("sandbox-manager", create_if_missing=True)
sb = modal.Sandbox.create(app=app)

p = sb.exec("python", "-c", "print('hello')")
print(p.stdout.read())
sb.terminate()