If you translate it literally, "per" is closer to "for".
If you don't translate it literally, I'd vote for "in itself". "In itself" (viewed in its essential qualities; considered separately from other things[0]) has a different meaning than "by itself" (alone/unaided). And to me it's clear that "per se" pretty much universally means the former.
A less literal translation like "essentially" or "in essence" is deployed by master Latin translators like Robert Fagles. I've even seen "in a vacuum" which does a better job at communicating the original intent than a string of cryptic prepositions.
If you don't translate it literally, I'd vote for "in itself". "In itself" (viewed in its essential qualities; considered separately from other things[0]) has a different meaning than "by itself" (alone/unaided). And to me it's clear that "per se" pretty much universally means the former.
[0] https://www.google.com/search?q=in+itself