conda is a completely no-go for me. It’s for data scientists, and only data scientists who do not need to care much about production.

Sadly this isn't always true, if you have non-pure Python dependencies, you might still need to use conda since they package database drivers or complex C/C++ dependencies for example.

SciPy already provides pip-installable wheels for a wide variety of platforms: https://pypi.org/project/scipy/#files