I don't understand why add another domain-specific command to a container manager and go out of scope for what the tool was designed for at first place.
The main benefit I see for cloud platforms: caching/co-hosting various services based on model instead of (model + user's API layer on top).
For the end user, it would be one less deployment headache to worry about: not having to package ollama + the model into docker containers for deployment. Also a more standardized deployment for hardware accelerated models across platforms.
(disclaimer: I'm leading the Docker Model Runner team at Docker)
It's fine to disagree of course, but we envision Docker as a tool that has a higher abstraction level than just container management. That's why having a new domain-specific command (that also uses domain-specific technology that is independent from containers, at least on some platform targets) is a cohesive design choice from our perspective.
they obviously didn't use vanilla postgres, but built some custom sharding on top, which is untrivial task (implementation and maintenance(resharding, failover, replication, etc)).
FYI, there is already a type of deployment called "sovereign cloud" where data exports are controlled by the country and already under works by major providers.
reply