Also, they can literally train the LLM inside the VM they create so it becomes an expert in whatever tools are there. And since they control the entire sandbox, less permission issues are likely.

I think they will become extremely vertical integrated sooner or later because that will be the most efficient and effective way to do this. I'm guessing anyone building this kind of thing now has only one hope: acquisition.