I think very soon big LLM providers (OpenAI et. al.) will provide a service like this. With the LLMs becoming faster, the agentic task completion bottleneck will soon move to the tool calls (both execution time and round-trip latency), so it makes sense to have a server host the project very close to the actual LLM doing the inference in order to minimize latency.
Also, they can literally train the LLM inside the VM they create so it becomes an expert in whatever tools are there. And since they control the entire sandbox, less permission issues are likely.
I think they will become extremely vertical integrated sooner or later because that will be the most efficient and effective way to do this. I'm guessing anyone building this kind of thing now has only one hope: acquisition.
Anthropic has Claude Code Web -- sorry, looks like the official name is "Claude Code on the web" -- which I think is basically this? I've been using it for a while and I really like it.
Nothing to do with latency, I think, it's all about convenience and safety. I don't have to worry about running an agent on the computer that knows all my admin API tokens and SSH keys, and the agent isn't interrupted if I close my laptop.
The one thing that might be useful is the ability to run on a beefier machine when you need it, e.g. make sure it has direct access to a GPU. But then you're back to thinking about the host VM, not fully abstracting away from it.
