PR #1029 · Code review · Request changes

Remote execd sandbox:
merge in two stages.

Execd is off by default, so this PR can merge once the 13 items that affect every deployment are fixed. Security, isolation and correctness items must be fixed before anyone turns execd on.

PR #1029 · feat: run agent code in a remote execd sandboxAuthor: michaluHead ffcfd08ff45 changed filesSandbox epic #353Reviewed: 27 Oct 2026

01 / Summary

What to fix, and when

Stage 1

Before merge

Changes that affect every deployment, even with execd off. Most are a few lines.

Items 1–13 · 0 high · 4 medium · 9 low

Go to stage 1

Stage 2

Before turning execd on

Security, isolation, the Manage flow and correctness under execd. Keep the Code sandbox section hidden or experimental until done.

Items 14–37 · 2 high · 15 medium · 7 low

Go to stage 2

Then

Questions and follow-ups

Three questions for the author, evidence for merge, and follow-ups owned by the author under the sandbox epic.

Items 38–40 · 6 follow-up groups

Go to questions

Open-source users: as written, this feature only works with sandbox-api, which is not public. So it does not help open-source users of CUGA yet. The design review follow-up covers how to make sandboxing useful for them.

What runs where when execd is on

The isolation applies to the agent's generated code, not to the tools.
Part	Runs in
Generated Python (CodeAct blocks)	Sandbox kernel, one per conversation, in a container shared by the agent's conversations
File tools and `run_command`	Sandbox kernel (separate copies of the tools, generated as Python)
All other tools: OpenAPI, MCP, knowledge, `find_tools`, `load_skill`, todos, spawned agents	CUGA process, called back through `POST /sandbox/tools/call`
Blocks in `cuga_lite_execution_mode = "function_calling"`	CUGA process, always

High data of other users or conversations at riskMedium wrong behavior, secret exposure, or a user-visible regressionLow limited impact, docs, or repo rules

Severity is the reviewer's estimate. Item numbers match the review comment on the PR.

02 / Stage 1

Before merge: affects everyone, even with execd off

Off until someone turns it on in Manage is the right default. When the sandbox is not turned on (or the box is ticked without Connect), nothing is saved, and the agent runs as on main, apart from these items.

Stage 1 findings
#	Severity	Finding and requested change
1	Medium	Startup must not depend on the sandbox settings row (`server/manage_routes/sandbox_routes.py:56`). If the DB is not reachable for about 3 s at pod start, CUGA does not start, even with execd off. Please log a warning and continue unless `sandbox_mode = "execd"`.
2	Low	The Helm chart fails for an empty `env:` block (`deployment/helm/cuga/templates/deployment.yaml:38`). When `env:` has no entries (null in YAML), `get .Values.env` fails with `wrong type for value`; main renders it, so `helm upgrade` breaks for such values files. Fix: `get (.Values.env \| default dict) $callbackKey`.
3	Medium	Do not expose `POST /sandbox/tools/call` when execd is off (`server/main.py:4688`). The route is registered in every deployment and has no user auth. Please return 404 unless `connection.execd_active()`.
4	Medium	Prompt change in `mcp_prompt.jinja2` (stop and report when `uv pip install` fails). It applies to every agent with the shell tool, not only execd. Please move it to its own PR, or state it in the description and share an eval run.
5	Low	"Remove all conversations" (`ChatLanding.tsx`) now clears the list before the deletes finish. If a delete fails, the list stays empty while the conversations still exist; on main they stayed visible. Please restore the list when a delete fails.
6	Medium	The Manage page shows the "Code sandbox" section to every user, also where no sandbox is available. Open-source users cannot use it, because it needs sandbox-api. Please show it only when a sandbox connection is configured (binding or settings), and mark it experimental until stage 2 is done.
7	Low	Description and comments. The description says "Under execd, all execution goes to the sandbox", but tool calls still run in CUGA, and function-calling blocks run locally. Please correct it and remove the tenant names. In the code, please remove or explain references that readers of this repo cannot follow (`sandbox/sandbox/README.md`, `execd-policy.yaml`, the `OCP/UBI9` comment).
8	Low	Empty default for `sandbox_secret_dir`, instead of `/bindings/cuga-sandbox`. `test_execd_connection.py:128-138` asserts this default, with the comment "The chart mounts the binding here", but the chart has no such mount. Please change the test together with the default.
9	Low	AGENTS.md test rules. `executors/filesystem/tests/test_backend_factory.py` is in a directory that no CI job runs; please move it to `executors/tests/` or add the directory to `unit-a`. `test_execution_policy.py:280`, `:291`, `:301`, `:306` have no pytest marker.
10	Low	`server/main.py:57` imports `execd.connection` at module top. Because `execd/__init__.py` imports `ExecdExecutor`, the server loads the whole execd module for everyone. AGENTS.md asks for new imports in `main.py` to go inside the functions that use them.
11	Low	A new INFO log line runs on every code block (`[execd:route] eval_with_tools_async ...` in `code_executor.py`). Please use DEBUG, or log it only in execd mode.
12	Low	`index.html` is now served with `Cache-Control: no-cache` (`serve_react` in `server/main.py`). The change is useful but not related to execd. Please move it to its own PR, or mention it in the description.
13	Low	The PR commits a rebuilt frontend bundle. If main gets frontend changes before merge, please rebuild it, or the bundle will undo them.

03 / Stage 2

Before turning execd on for any users

Turned on, then off, new conversations run as on main again. The items below matter as soon as execd is turned on.

Security and isolation

Security and isolation findings
#	Severity	Finding and requested change
14	High	Document the isolation model. All conversations of an agent, also those of different users, share one container, and a conversation is only a directory. With the shell tool on, `run_command('cat ../<other-thread>/uploads/x')` reads another conversation's files, even after item 20 is fixed. The kernel also runs agent code with full builtins and free imports, while `LocalExecutor` adds restricted imports, safe builtins and `assert_safe_globals` at runtime; under execd, only the static `SecurityValidator` check remains. Please state this in the docs and in the Manage text, so operators do not turn on the shell tool for agents with several users until this is addressed.
15	High	Two thread IDs can share one workspace folder (`execd/execd_executor.py:365-372`): `d57c950e.aaaa-…` and `d57c950e-aaaa-…` map to the same folder, so one user can read and change another user's files through the workspace API. Please use one collision-free mapping everywhere.
16	Medium	Changing the URL keeps the saved API key (`server/manage_routes/sandbox_routes.py:119`). With a new URL and an empty key field, the saved key is kept, and the probe sends it right away to the new host in `X-API-Key`; after that, all agent code goes there. With the default Helm values (authorization off), any logged-in user has Manage access. The probe also returns `status_code` and `probe_error`, which can be used to scan the internal network. Please ask for the key again when the URL changes, and return only a short status from the probe.
17	Medium	Do not write the env API key to the DB (`server/manage_routes/sandbox_routes.py:119`). When no row exists and the key field is empty, the raw `execd_api_key` is saved in `sandbox_settings`. Please store only references (`db://`, `vault://`, `env://`).
18	Medium	Keep kernel helper aliases away from agent code (`execd/execd_executor.py:940-943`). `_cuga_os`, `_CugaPath`, `_cuga_glob`, `_cuga_sh_os` and `_cuga_cb_req` pass `SecurityValidator`, so agent code can read other conversations' files, or run shell commands with the shell tool off. The smallest fix is to add these names to the validator; in the kernel it is the only check (item 14).
19	Medium	Tool approval can be skipped through the callback helpers. `await _cuga_cb_acall('send_' + 'email', [...], {})` calls any tool of the block without its name in the code, so ToolApproval, which looks for the tool name as text (`policy/agent.py:1393`), does not trigger. Please keep `_cuga_cb_call` / `_cuga_cb_acall` out of the agent's namespace too.
20	Medium	Confine `/workspace` in the kernel `run_command` (`execd/execd_executor.py:895`), the same way the host version does. Today `run_command('ls /workspace')` lists other conversations, and `run_command('python /workspace/x.py')` cannot find a file the agent just wrote with `write_file('/workspace/x.py')`. This prevents mistakes; it is not isolation (item 14).
21	Medium	Limit the callback route when execd is on (`server/main.py:4688`, `execd/tool_callback.py:101`, `:157`). It is unauthenticated and reads the whole body before it checks the token, so very large POSTs can exhaust memory, and a non-ASCII `block` or `token` causes a 500 with a traceback. Please cap the body size (results are already capped at 8 MB) and refuse non-ASCII values.
22	Low	Escape the thread ID in the release URL (`execd/execd_executor.py:831`). Thread IDs are free text, so deleting a conversation named `victim#x` sends `DELETE /threads/victim` to sandbox-api (httpx treats `#` as a fragment), and the victim's kernel is lost. Please use `quote(routing_id, safe="")`.
23	Low	The package index URL is logged at INFO (`execd/execd_executor.py:389-395`) and exported into kernels and shell commands. If it contains a token (for example `https://__token__:<PAT>@…/simple`), the token leaks into logs and into agent code. Please remove credentials from it before logging, and document that it must not contain any.

Manage on/off flow

Manage on/off flow findings
#	Severity	Finding and requested change
24	Medium	Turning it on saves before checking (`server/manage_routes/sandbox_routes.py`, `put_sandbox_settings`). It saves `enabled=true`, then probes. With the cluster binding, ticking the box saves at once; with manual settings, Connect saves at once. If the sandbox does not answer or the key is wrong, it stays on and every block fails until someone turns it off. Please save only after a successful probe, or roll back on failure.
25	Medium	A missing callback URL fails every block that has tools (`execd/execd_executor.py:1151`). Helm sets `sandbox_callback_url`; without Helm (local, or other deployments) it is empty, so every block in an agent with tools fails with "Tools cannot be called from the execd sandbox". Please check it in the probe and show it in the Manage page.
26	Low	The callback fallback can point at the registry. When `sandbox_callback_url` is empty, `function_call_host` is used, which the docs describe as the registry URL (port 8001). `/sandbox/tools/call` exists only on the main app, so every tool call then gets 404.
27	Medium	Opening the Manage page can drop the API key for running conversations (`server/manage_routes/sandbox_routes.py:158`, `execd/connection.py:121`). The page calls refresh on every load when the sandbox is on. Refresh clears the key cache before the DB read, so a request in that gap caches an empty key and gets 401. If the DB read fails, the empty key stays until a successful "Test again". Please load the new key first and then replace the cached one.
28	Medium	A failed on/off request leaves the checkbox in the wrong state (`SandboxSettings.tsx:85-89`). The box changes before the request and is not restored on error; the only sign is a toast that disappears after 5 s. An admin can think the sandbox is off while it is still on. Please restore the box from the server state on error.
29	Low	The first Connect saves the env URL in the DB, and the UI has no reset (`server/manage_routes/sandbox_routes.py:112-118`). A later change of `EXECD_URL` is then ignored without a warning, and a saved key cannot be removed from the UI (`clear_api_key` is never sent). Please add a reset and a "remove key" action, or do not store values that came from the env.
30	Low	Turning it off leaves the kernels of existing conversations in the sandbox (release is skipped when execd is off), and files written there are no longer in the workspace panel. Those conversations then continue on the host without their files. Please release kernels when it is turned off, or document it.
31	Low	The switch is global for the instance, and kept in process memory. One admin turns it on for all agents and users. With more than one replica or worker, only the process that received the request changes. Please say this in the Manage text or the docs.
32	Low	Settings that bypass the toggle. `sandbox_mode = "execd"` with no URL or secret fails every block, and the Manage page shows no error (`connection.status()` reports no error for an empty URL). `execution.python_backend = "execd"` without the toggle (`execd/execd_executor.py:121`) sends Python to execd with no key, while file tools and the shell stay on the host. Please note in `settings.toml` that the Manage toggle is the supported way to turn the sandbox on.

Correctness under execd

Correctness findings
#	Severity	Finding and requested change
33	Medium	The kernel `run_command` hides failures (`execd/execd_executor.py:906-922`). A failed command with empty stderr is returned as success (no exit code), and there is no timeout or kill. The host version reports `(exit code N)` and stops after 120 s.
34	Medium	Kernel `edit_file` and `read_file` (`execd/execd_executor.py:1085`, `:1007`). `edit_file` checks matches on the original text but applies edits to `new_content`, so a chained edit can fail, or do nothing and still be reported as applied. `read_file` with `grep_pattern` prints `LINE\|` instead of the line number. Both are one-line fixes.
35	Medium	Kernel file tools raise exceptions where the host tools return error text. `move_file`, `search_files`, `get_file_info` and the path-escape check raise, while the host tools return `[<tool> error] ...`. Under execd, `await get_file_info('missing.csv')` fails the whole block and drops its variables. `get_file_info` also returns 5 fields instead of 8.
36	Medium	Any tool named `run_command` is replaced by the sandbox shell (`execd/execd_executor.py:1127`). File tools are matched by the `_cuga_app_name` tag, but `run_command` only by its name. An SDK or MCP tool called `run_command` is never called under execd, and shell commands run in the sandbox even with the shell tool off. Please tag the shell tool and match on the tag.
37	Medium	A block that never reports a result is shown as a success (`execd/execd_executor.py:1199-1203`). If the kernel dies or the stream ends early, the agent gets its partial output or `<code ran, no output printed to stdout>`, with no variables and no error. Please return an error that says the block did not finish.

04 / Questions

Questions for the author

Questions
#	Severity	Finding and requested change
38	Low	Scope headers before the first code block. `X-Tenant-Id` / `X-Agent-Id` come from a cache that is filled only when a code block runs (`execd/execd_executor.py:174`). Uploads before the first message, the workspace panel after a restart, and file tools in function-calling mode send `X-Agent-Id: cuga-default`. Does sandbox-api route these to another agent's scope? If yes, the first kernel and the uploaded files land in the wrong place.
39	Low	Packages in the sandbox image. Agent code now imports from the sandbox image, not from CUGA's environment. The import allowlist permits `pandas`, `numpy` and `pydantic`. Does the image contain all of them, so that blocks that work locally also work in the sandbox?
40	Low	Reaching other kernels. execd and all kernels run in one container. Can agent code reach execd's local API, or other kernels' connection files? If yes, one conversation could run code in another user's live kernel and use its tool token.

05 / Evidence

Evidence for merge

Please post a short manual run from your environment with execd on: 2–3 conversations at the same time on demo_crm, each finishing its task, plus one conversation delete that releases its kernel. This is just to validate that we tested at least the out-of-the-box agents with execd, like the knowledge agent. Maybe you already did this; I'm relying on you for this check.

Please also run demo_crm once on main and once on this branch with execd off, and confirm the results are the same. Run it once more on this branch with the sandbox connection configured (URL or secret) but not turned on in Manage, and once in a new conversation after turning the sandbox on and off again. Both should also match main.

06 / Follow-ups

Follow-ups

Please open these as sub-issues of the sandbox epic #353, assign them to yourself, and link them on the PR before merge, so they are tracked.

Support public OpenSandbox execd

auth header, release route, lost-kernel detection without body text. Add a docker compose or Kubernetes setup for people in the open, or for local testing.

Design review

we need to set up a design review meeting to consolidate the sandbox interface or API into one sandbox interface for all backends, and one file-tool implementation. A proper design lets the user choose the sandbox backend, for example pymonty, local Python execution, OpenSandbox / public execd, or any service that implements a generic sandbox API contract. sandbox-api is then one implementation of that contract, not the only option. This is what makes sandboxing useful for open-source users. Each backend declares what it supports (Python, shell, files, isolation per conversation, tool callback). The shared parts (tool callback, file tools, path confinement, variables, call budget, workspace panel) are built once on top of the interface. Related: #354 (which code paths run inside the sandbox) and #997 (Pydantic Monty backend).

Isolation

apply the runtime restrictions of LocalExecutor in the kernel (restricted imports, safe builtins, assert_safe_globals), and use a sandbox per user or per conversation for agents with several users.

Lifecycle

Keep the thread→kernel map across restarts. Today, after a restart the workspace tree is empty, and deleting a conversation leaves its uploads in the shared sandbox.
Remove /workspace/<thread> after delete.
Spawned agents with share_workspace=True do not see the parent's files in code blocks (the kernel uses the child's thread ID, execd/execd_executor.py:1117), and their kernels are never released.
Supervisor variables are lost between steps under execd (execd/execd_executor.py:1114): step 2 cannot use step 1's result.
Interrupt the kernel when a block times out (execd has DELETE /code). Today, blocking code keeps the kernel busy, and later blocks in that conversation fail.
Tool calls already running in CUGA keep running after the block times out or the user presses Stop (the stub uses run_in_executor, and registered_block only removes the entry), so a slow create_order still creates the order after "Execution timed out".
A failed kernel bootstrap is only logged; the context is kept and bootstrap is never retried, so the working directory and venv path are missing for the whole conversation (execd/execd_executor.py:472-477).
Evict idle kernels and remove each conversation's .venv. Today, disk and memory grow with every conversation.

Tests

all new tests use a sandbox-api mock, and the generated file-tool and shell code in the kernel is never run (one test only checks that the text async def write_file appears); the tool-callback stubs are tested against a local server. Add tests that run the kernel tool code in a real Python process, and one live e2e or manual test that runs several conversations at the same time against a real execd. Until then, the manual run under Evidence for merge is enough.

Smaller items

A tool result with NaN or Infinity returns HTTP 500 after the tool already ran (JSONResponse refuses NaN; execd/tool_callback.py:177), so a retry can repeat a side effect.
JSON type changes between blocks (datetime becomes a string, int keys become strings, tuples become lists).
One context per tool callback (concurrent calls share one contextvars.Context).
Skills tarball size (one env string, 128 KiB limit).
thread_id=None creates throwaway kernels that are never released.
Bootstrap time counts inside the callback window.
Helm secret mount for the sandbox binding.
Single-replica note.
Function-calling replies report the execd timeout.
Argument validation (non-JSON arguments silently become strings).
"Did you mean" hint missing on the execd error path.
Settings table mismatches in the description.
Typed URL and key are erased when a check finishes (SandboxSettings.tsx:42-47).
Binary files are previewed as replacement-character text (errors="replace", execd/filesystem_backend.py:69).
A task left running from one block calls tools with the next block's token, contrary to the tool_callback.py docstring.
Performance: the workspace tree lists the whole .venv on every poll, all variables are sent again on every block, and uploads go in 48 KB chunks.
The kernel release on delete uses the 120 s block timeout, so with the sandbox down, each delete can hang for up to 120 s.