I'd still be interested, if and only if, the "wake word" processing is done locally and it THEN sends the recording that occurred after the wake word up to the cloud. If it is the case that even the wake word processing is done in the cloud, non-starter for all the reasons you state.
I know for a fact that the wake word processing is done locally on the device. You could even check its network traffic to see that this is the case.
I'm not sure what protections you would have against a secret court order making the device always listen (via an OTA update) for select individuals, but you can at least check that they don't have such monitoring enabled for most devices.