Isn't it just a bandit problem? Couldn't you use something like Thompson sampling to sort the comments? It'd have the nice benefits of adding random noise like reddit tries to do, while still having finite-time optimality guarantees.