Whenever HFT comes up on HN, someone asks why we don't just remove the incentive for low latencies by matching orders less frequently. It's a fair question, but I think there are good reasons not to do this.
Let's say we discretize the exchange to, say, 1 Hz, so all the orders are queued up and then executed simultaneously at the next clock tick. Now, on a given tick, there will almost always be a mismatch between the number of buy orders and the number of sell orders, so some orders will go unfilled. How do we choose which ones? (For this discussion I am ignoring price. Obviously we will fill more aggressively-priced orders first; the question is how to prioritize orders of equal price.)
We could give priority to the ones that arrived first, but of course then we're back to traders racing each other. The only other scheme I can think of [1] is to fill every order in proportion to its size. So if Alice wants to sell 200 shares, Bob wants to buy 100 shares, and Charlie wants to buy 300 shares, we give 50 shares to Bob and 150 shares to Charlie.
Well, this scheme has its own problems -- arguably worse ones than the current system! If Bob really wants to buy 100 shares, and expects to be competing with Charlie, he has an incentive to place a much bigger order: in this case, if he knew Charlie's bid size, he would also ask for 300 shares, expecting to get 100. Of course, Charlie will be playing the same game and inflating his own bid size. The equilibrium is that everyone asks for way more size than they actually want.
There are two things wrong here:
1. Traders are spending mental energy trying second-guess each other's order size; those who don't are crowded out of the most competitive (and, typically, profitable) trades.
2. Since traders can't guess correctly every time, they will sometimes end up buying or selling much more than they want, which means they have to turn around and do the opposite trade, paying the spread in the process. If this happens during during a large price movement, these traders can lose even more money, and in their haste to reverse their trades, they will drive the market even further in the same direction.
And by the way, this is not just speculation. While I'm not familiar with any exchanges that discretize their clocks this way, there is a product with a similar system: the CME Eurodollar contract [2]. In this contract, unlike the other futures products traded on the CME, resting orders do not execute on a first-come-first-served basis; instead they are allocated "pro rata", or in proportion to their size, much as I described above. Based on my conversations with several people who trade them (including HFTers), the results are also as I described: traders routinely over-order and suffer the consequences.
Hopefully this sheds some light on why this seemingly obvious solution is not widely implemented.
[1] Of course I can think of many other schemes, but they either have obvious problems or basically reduce to this one.
[2] There are other products matched this way, but I'm not as familiar with them.
Why not use a random order for order fulfillment at each tick? No skewed incentives with that rule.
If that creates too many concerns over security of the RNG, then use deterministic rotating ranks. Traders may be able to predict who they will be able to beat out on the next tick, but the high ranks rotate throughout the herd, so everybody gets a fair shot at beating out others at the same price.
Hmm. So let's say we assign every trader a unique ID and randomly (or not) prioritize the IDs every tick. (I'm assuming you're shuffling the IDs and not the individual orders; otherwise, as jsnell points out, it averages out to the pro rata case.) The main problem I can see is that if someone can get their hands on multiple IDs, they can duplicate their orders across all their IDs and we're back to the pro rata system.
Even if you make sure to assign just one ID per trading firm, firms could coordinate to share their IDs. For example, if you're not using your ID this tick, you might place an order on my behalf for a small fee. And I'll also have sent some of my orders through an independent broker to take advantage of their ID. (Of course, if that broker places many such orders, and only some of those fill, they'll need some policy for which of their customers get the trades... maybe they could use time priority?)
It's conceivable (if unlikely) that the SEC could enact and enforce a ban on such collusion. Even then I think there would be other perverse incentives, but I'll need to think more about it.
A random ordering will create an incentive to split orders into smaller pieces. Which at the limit is going to devolve to the the proportional fulfillment case once everyone submits only orders of the minimum size.
I was gonna post a comment saying what they meant was obvious, but then I just went "ohhhh." * That's actually quite an annoying little problem there.
One work-around might be to add rules to the exchange forbidding the placing of multiple bids by the same entity at the same price, and then allocating randomly.
This is also reminding me of the Talmud's descriptions of how to pay off debtors when there isn't enough money to go around: http://mindyourdecisions.com/blog/2008/06/10/how-game-theory... Such a system could work for exchanges too, although it still encourages over-bidding.
Surely also over-bidding has its own risk/reward curve so it would be reasonable to allocate pro-rata and let people work out how much to over-allocate.
What you see as "an annoying little problem", I see as a sign that the system is consistent and robust. It's a good thing that the system doesn't distinguish between a hundred one-share orders and a single hundred-share order; such a distinction would be totally arbitrary. (By contrast, ranking orders by time is not arbitrary. All else being equal, it's better for things to get resolved faster; the only question is whether we're giving too much of a reward for too little of an improvement.)
In general, we want traders to spend their time thinking about asset prices and risk, not market structure and game theory. The price-time priority system is a very simple one that rewards traders for deciding what they want and then announcing it right away. All the modifications that have been proposed in this thread encourage traders to play games, second-guess one another, or otherwise work around the system.
There are definitely tradeoffs, but personally I think it's better to use the cleaner system and accept the latency arms race than to add a layer of artificial incentives -- and for what it's worth, it appears that nearly every major electronic market has come to the same conclusion.
Finally, thanks for the link to the Talmud article: it was a very cool application of game theory to history. (Of course, I don't see any indication in the article that the Talmudic system would be an improvement over pro rata.)
Just referring back to this from the comments on the second part, and just wanted to mention I meant "annoying little problem" in the sense that it seems a shame there can't be other solutions, like how Arrow's Impossibility Theorem prevents achieving a perfect voting system (http://en.wikipedia.org/wiki/Arrow%27s_impossibility_theorem). I definitely see what you mean about this being the most optimal solution from the point of view of actual pricing and risk. The downside though is there is profit to be made in slicing time every quicker which can't be prevented without breaking something else, but the money spent on this time-slicing is broadly wasted because it is just an arms race.
So I don't think the current solution is necessarily wrong; just annoying we can't get all the benefits and also avoid the arms race.
No problem, I very much like the Talmud article and how simple the explanation works out to being.
Yeah, that and SEC Rule 612 (no subpenny prices, as the article mentions) both make bucketing sensible. Have a (1s,$0.01) granularity and HFT pretty much goes away, the National Best Bid/Offer makes sense again, and flash crashes only happen once a decade. If regulators actually cared about about market stability, they'd be pushing for something like that...
The London Metals Exchange - being an outlier as always - has gone for a different approach: they throttle traffic to 40 actions/client/second and smooth it out over as many seconds as necessary - so a 10,000 quote burst will be smeared over 4 minutes... good luck trying to manipulate the market with that, HFT boys.
Every market has a book, and a price. If a sell comes in at under the price, and there are unfulfilled buy orders anywhere above the price, the sell executes at that price for the order that has been above that price the longest. Conversely if a buy comes in above the price and there are unfulfilled sell orders below that price, the trade executes at that price for the sell order that has been below that price.
The price itself drifts upwards or downwards at a set rate depending on whether there are currently an excess of buy or sell orders that would execute at the current price.
the scheme you're describing is no different than the continuous case. whoever came in first gets filled first. it doesn't matter if matching happens at a set interval, getting into the book first still matters, and HFT continuous as usual
You can do it based on time. Yes it there will still be some incentive to be faster, but the incentive will not be that great. And you can remove all incentive to be faster by simply increasing your buy price by a penny. This means that all the incentive for speed is less than one penny per stock.
This would result in much smaller incentive for speed and less money and energy spent on being faster, and more fair markets, etc.
Sorry, I meant to say "the every-N-seconds auction is not a good idea, even with the itayose method". The reasons are basically the ones traversal mentioned above.
Let's say we discretize the exchange to, say, 1 Hz, so all the orders are queued up and then executed simultaneously at the next clock tick. Now, on a given tick, there will almost always be a mismatch between the number of buy orders and the number of sell orders, so some orders will go unfilled. How do we choose which ones? (For this discussion I am ignoring price. Obviously we will fill more aggressively-priced orders first; the question is how to prioritize orders of equal price.)
We could give priority to the ones that arrived first, but of course then we're back to traders racing each other. The only other scheme I can think of [1] is to fill every order in proportion to its size. So if Alice wants to sell 200 shares, Bob wants to buy 100 shares, and Charlie wants to buy 300 shares, we give 50 shares to Bob and 150 shares to Charlie.
Well, this scheme has its own problems -- arguably worse ones than the current system! If Bob really wants to buy 100 shares, and expects to be competing with Charlie, he has an incentive to place a much bigger order: in this case, if he knew Charlie's bid size, he would also ask for 300 shares, expecting to get 100. Of course, Charlie will be playing the same game and inflating his own bid size. The equilibrium is that everyone asks for way more size than they actually want.
There are two things wrong here:
1. Traders are spending mental energy trying second-guess each other's order size; those who don't are crowded out of the most competitive (and, typically, profitable) trades.
2. Since traders can't guess correctly every time, they will sometimes end up buying or selling much more than they want, which means they have to turn around and do the opposite trade, paying the spread in the process. If this happens during during a large price movement, these traders can lose even more money, and in their haste to reverse their trades, they will drive the market even further in the same direction.
And by the way, this is not just speculation. While I'm not familiar with any exchanges that discretize their clocks this way, there is a product with a similar system: the CME Eurodollar contract [2]. In this contract, unlike the other futures products traded on the CME, resting orders do not execute on a first-come-first-served basis; instead they are allocated "pro rata", or in proportion to their size, much as I described above. Based on my conversations with several people who trade them (including HFTers), the results are also as I described: traders routinely over-order and suffer the consequences.
Hopefully this sheds some light on why this seemingly obvious solution is not widely implemented.
[1] Of course I can think of many other schemes, but they either have obvious problems or basically reduce to this one.
[2] There are other products matched this way, but I'm not as familiar with them.