One of HFT’s objectives has always been to make the market more efficient. Speed traders have done such an excellent job of wringing waste out of buying and selling stocks that they’re having a hard time making money themselves.
One of its objectives?
Those philanthropic high-frequency traders just wanted to make the market more efficient for everyone? Sure they did. Sure they did.
You can say that about any industry that turns a profit. Those philanthropic $MEMBERS_OF_MONEY_MAKING_INDUSTRY just wanted to make $THING_INDUSTRY_DOES better for everyone? Sure they did. Sure they did.
Making money is not a character flaw. Otherwise, many prominent members of this board would be horrible people.
Your caveat doesn't apply if they aren't helping markets be more efficient. HFT is basically a latency arms race ripoff scam. Whoever pays more for faster lines and closer access to the datacenter takes the cake. Thousands of PHDs are working on this instead of real stuff.
Say this is the makeup of market participants: all of them are normal merchants buying fruit for their shops but 2 are latency arbitragers. Shipments of fruit don't always come in because sometimes the sail boat crashes into something.
The latency arbitragers hire high speed skiffs to go scout for the ship before it arrives. If one of them sees that a shipment of oranges are missing, he rushs back to game the unsuspecting others in the market: orange prices are going to go up so he buys them on the cheap.
Because nothing gets better with two competing arbitragers. They simply compete with each other on hiring out more skiffs and paying for elaborate faster ones. Or signaling by latern along a chain of ships.
None of this helps the market; all other participants lose out. With competition between the two arbitragers they end up wasting most of the money buying faster ships. Ships that could be used for.. rescuing crashed orange ships? Many other things as well. The worst government bureaucracy couldn't be more wasteful.
Markets aren't efficient even in theory if there is an information asymmetry.
I hope you can see the analogy between my example and things like:
* deploying more and more servers to literally turn our energy into waste heat (they could be simulating protein fold or something)
* paying trading exchanges extra for in-house servers (this is just a complete scam)
* Digging hundreds of miles of new fiber line to save 2ms of latency over the existing line (with no plausible consumer benefit because the old lines had plenty of dark fiber and the new line is monopolized by whomever pays the most)
* Setting up microwave (lantern?) relay towers to get from New York to Chicago a millisecond faster
* Making markets more likely to have a major crash by forcing developers to use C++ to eek out a couple milliseconds over safer alternatives
You can make a little bit better case for electronic market making.
You are conflating market making and latency arb, which are not the same thing. All of your points apply to latency arbitrage and you haven't said anything about market making.
As long as there are profitable arbitrage opportunities, people are going to go after them. The profits from latency arb aren't infinite and things will hit an equilibrium when the profits line up with the costs. Incidentally, I don't think latency arb is a bad thing. The arbitrageurs keep prices in line between exchanges, so you don't have to worry as much about getting a crappy price at one exchange when a better price was available at another exchange.
No I'm not, I clearly said "You can make a little bit better case for electronic market making." But if you think market making has no major latency component, you are living in the 70's.
Look at what happened to knight, it could have spilled out to the whole market and they would have had to roll back trades or something (I'm not blaming knight's thing on C++).
That's my point though! Rolling out the wrong systems, bad algorithms etc etc could happen with any language.
If there was a memory corruption, or a security error or something, I could see it being attributable (in part) to the language. The knight thing seems more of a human cock-up.
Let me make it more specific: it is soft-realtime C++. It is way more error prone and on top takes way longer to wright code with same behavior as regular C++ (ignoring latency), no to mention it takes much longer to develop.
I think they mean "more efficient" like the cotton gin made the cotton industry more efficient. By reducing the labor involved in processing cotton, the companies that used cotton gins could eek more profit from every acre of production.
HFT doesn't "wring the waste" of inefficiency back into the market, but into the hands of the HF trader. HFT does benefit normal market traders, but I'm not sure to what extent.
What they're actually saying is that there's now too many high-frequency traders competing for there to be much profit left in this money scraping business.
"When the market opened on Aug. 1, a new piece of trading software that Knight had just installed went haywire and started aggressively buying shares... By the end of Aug. 2, Knight had spent $440 million unwinding its trades"
I'd love to know more about that particular launch. A hole in somebody's test suite!
One of the prevailing theories from http://www.nanex.net/ is that when Knight went live they also put their order simulator live as well.
The thinking behind this theory was that a simulator would keep adding liquidity to a fake market for testing a market making algo.
During the day that Knight lost their pants nanex saw that while their algo was taking liquidity someone else was constantly providing it, ie the bid ask spread didn't really open up that much and the price held relatively flat.
This meant that someone else was constantly adding liquidity that Knight was taking away. The thinking was that Knights market simulator was the one sending out the live orders to add liquidity.
TL/DR Knight crossed themselves many many times in a few minutes.
As an example of why Nanex is worth listening to, here is an article [1] describing how Nanex claimed Monday's ISM manufacturing data was leaked 15 milliseconds early, leading ultimately to an admission by Reuters that it caused the leak. Eric Hunsander, a software developer at Nanex, went so far as to hint to Reuters that they should check their NTP settings since a drift of 15 milliseconds is allowed by default [2].
Eric is very interesting to follow on Twitter [3]. He highlights mysterious HFT related events on a daily basis. My only complaint is that his articles and charts, in particular, are very difficult to interpret. As an outsider to the HFT industry, I don't know where to find the resources to learn enough to interpret most of Eric's insights.
... claimed Monday's ISM manufacturing data was leaked 15 milliseconds early ...
While I'm peeved that as an adult I'm still not able to purchase an affordable flying car (though I was able to afford a tiny flying robot recently), there are days where I do think I am living in the science fictional future anticipated in my youth. Reading that CNBC article... this is one of those days.
The thinking behind this theory was that a simulator would keep adding liquidity to a fake market for testing a market making algo.
If you're testing a market making algo (i.e. to participate in NYSE's RLP program), you're simulator needs to remove liquidity, not add it. The idea is that the simulator (the tester) accidentally got deployed, and then went around crossing the spread (removing liquidity) and building up large positions in various instruments.
TL/DR Knight crossed themselves many many times in a few minutes.
A self trade doesn't affect your profit. Any self-trades would have been problematic because it's illegal to paint the tape, but that's not what caused the losses.
NYSE's RLP program is for consumers; market makers my be providing liquidity for it, but certainly aren't taking (except if they handle customer orders as well). Market Makers by definition tend to provide liquidity; most are obligated to keep a bid and offer on each stock they make a market in available at all times, and they get a few benefits like not having to locate shorts.
My point was that if you are developing a program to do electronic market making (adding liquidity), then the program you write to test that program must by definition be taking liquidity (it shoots test orders at the quotes provided by your market making algo).
applying occam's razor, i'm not sure which would be the simpler explanation:
1. a test suite got deployed accidentally, that happened to exactly match the buying predisposition of the production algorithms and fronted a bunch of their own trades @ a loss, or
2. someone got greedy and spilled the beans, and a competitor knew which algorithm would be unleashed on day X and buried it with their own
Nanex has lots of fun data on the Knight incident:
"We believe Knight accidentally released the test software they used to verify that their new market making software functioned properly, into NYSE's live system."
Nanex is a company that intrigues me immensely. I follow Scott (their founder, I think) on twitter and I wish I understood even a small percentage of what they produce on a daily basis. The stock market in general is something I wish I understood more of. Can anyone recommend some reading material?
From what I read about this crash the most important lesson was to have a kill switch. The biggest problem Knight Capital had was that they could not switch the system off quickly. It was roughly 45 minutes (according most accounts) between Knight realising something was wrong to ceasing trading.
That there was a bug in some code isn't surprising. That it was so hard to stop their trading is really surprising.
In Knights defense they aren't just a high frequency hedge fund that trades their own book to make a profit.
They have legitimate customers that they put orders out on behalf of and allow clients to trade through them.
Part of the reason why it took so long to shut the system down is that if they had just shut the system down, then they'd be in breach of contract for those clients.
So shutting down was more of a business decision that a technical one.
IIRC there were indications that they restarted the system twice in those 45 minutes, to no avail. Presumably it took that long to get someone with the authority to make a decision to switch it off entirely to decide to do so.
apparently there was a test component that took every order the model generated and copied the message 5 or 10 times - supposedly to stress test the system. Somehow this component was in production. Real orders were going out to the market and executions flowed back to an error account that was not being monitored. I imagine if you were monitoring the PnL you would have seen massive profits in the first few moments that the system went haywire - only to realize that you were simply just driving up the prices of your own positions...
I think the story goes that something not ready for production made it into production. I obviously can't say anything about their test suite, but it sounds more like a mismanaged merge/deployment process.
I had initially heard that the test program was released to production causing the system to take the wrong side of every trade, but I don't think it was ever confirmed.
HFT basically involves an arbitrage. The more competition there is, the less profits are made. It's simply a case of the market becoming more efficient, so the arbitrage opportunity is gone.
Meanwhile, consumers of market makers (ie. normal investors) benefit from being able to make any trade in a split second, for very little cost. In the end, it still requires a human element.
"According to Rosenblatt, in 2009 the entire HFT industry made around $5 billion trading stocks."
Where does this money come from? The industry doesn't actually create any wealth. There must be a greater fool who lost this $5 billion to the HFT industry. But who is that? Who became $5 billion poorer to make the HFT guys that $5 billion richer? Is the article just counting the total profit of HFT winners and ignoring the HFT losers?
This is not rhetorical, I really want to understand this.
HFT is a way for market makers to compete among each other. When I want to buy stock in some company or other I need someone to sell it to me, and when I want to sell a stock I need someone to buy it. I might get lucky and someone like me might be looking to sell or buy at the same time, but that's unreliable. However, there are people who have some of pretty much every stock out there, and who are always ready to sell a share of ACME to me for $11.59 or buy it from me for $11.58. Because there's a granularity to the prices of stocks the market makers can't compete on price, so exchanges end up using speed to resolve who gets the trade. Hence HFT.
As in most things of this nature, I think a good deal is to be learned by who are the most vocal informed opponents. The most vocal market participants against HFT are usually large, institutional market makers who used to grow fat on $1/16 spreads and liquidity rebates (Themis, I'm looking in your direction). These guys used to not have to work particularly hard for the cash, but they're getting their lunches eaten by new market participants who can do it faster and cheaper and have built their companies around integrating new tech very rapidly.
You'll find people trotting out the same old chestnuts (brain drain, unfairness of server colocation, etc.) but those are distractions from the interesting issues.
As an aside, I once had the distinct pleasure of watching a conversation on one of the CNBC programs running in our kitchen in which a NYSE floor trader actually used the phrase "we're the institutional investor looking out for the little guy" while managing to keep a straight face. This dude probably would have curb stomped his own grandmother to pick up a $100 bill. Even the usually sympathetic hosts just laughed outright.
I'm surprised that the outrage against HFT is so strong given how comparatively little is at stake there. After the HFT's much, much, much larger financial cousins, the "big bank" market participants nearly destroyed the entire planet's financial system, it's quite a coup for them that they've shifted the outrage to the guys competing over less money than some banks pay out in bonuses.
I have no love for anyone in the financial industry (I'm a self-loather, if you haven't noticed), but the HFT guys are way, way further down on my list of people to be pissed off at.
TLDR: ... their stated value add of liquidity does not compensate the US and World Economy nearly enough for the risk of collapse they introduce into the system.
I've read this a few times over the last year, and I still don't know how I feel about it. He's obviously a savvy businessman who knows his way around the equity markets a bit, but I can't shake the feeling that he's confusing a lot of issues and ignoring the more pressing ones.
Banning HFT or otherwise regulating them out of business will have about as much of an effect at "bring[ing] our markets back to their original goals of creating capital for business" as banning Super Extra Gulp sodas would have in curbing the obesity epidemic. (Incidentally, that quote about the true purpose of a market is a claim I've never seen proven, and even were that true, it would be only true in the case of equity markets, which are not the only kind.)
If one accepts that the financial industry is a perversion of its original goals (whatever they may be), I can think of a lot more effective places to start than the HFT guys: the bond ratings framework that nearly helped lead us to economic disaster; the outsize influence that big banks (i.e. not HFT shops) have in our political climate; the irrational insistence that unregulated financial markets are "best"; etc.
Blaming the HFT guys seems to be an easy out rather than tackling the more difficult issues. Judging by the regulatory and legislative response to the crisis (or more accurately, the lack of an effective response), that's true. It's a lot easier to blame a technology that isn't well understood by outsiders than to reform the political election funding system, or to recognize that it is an absolutely mind-numbingly stupid idea to have bond issuers pay to have their own bonds rated and allow them to choose what company rates them.
Yes, the flash crash and subsequent events have caused some concern, but these events corrected themselves relatively quickly. It's an issue, but the far bigger issue is that we haven't effectively dealt with the thing that very, very nearly put us in a second great depression.
The finance industry creates wealth by making financial transactions and investment instruments cheaper and more efficient.
Same way as a rail company or Amazon creates wealth not by 'building things', but by moving things from point A to point B cheaper, more easily and for less risk.
But there's still a loser there: the older transportation or retail providers who can no longer sustain their high profit margins. The Pennsylvania Railroad took in profits that used to belong to Erie Canal shippers. Amazon got rich by collecting money that used to go to Barnes & Noble at brick-and-mortar retail. So from where did the profits or money come that make up the HFTs' profit?
From the other responses, it sounds like the real answer is salami slices from lots of other market participants. A large mutual fund buyer pays a slightly worse price because the HFT jumped in ahead of them to arbitrage between the selling market maker and this buyer. So the HFT's profit comes out of the mutual fund's capital. The loss would show up as tracking error for an index fund, for example.
I don't think that's correct. We have always had market makers: people who trade purely for profit. They grab stocks in anticipation that others will want them at a higher price. So, if I own stock, and you own stock, instead of the stock going straight from me to you, it's more likely that there will be a market-maker inbetween us.
The benefit of this is that the chances are small that you and I will be trading at exactly the same time. So, I say "I want to sell this stock," and a market maker buys it. Later, you say "I want to buy that stock", and you buy it from the market maker. The market maker makes money on the profit of the sale. So, I may get a little less, and you may pay a little more. But, it means that when I wanted to sell, someone was there. And when you wanted to buy, someone was there. That's what people mean by liquidity.
So, back to HFT: the losers are the conventional market makers. The guys in the pit who used to shout-out buys and sells.
The economy is, thankfully, not a zero sum game. The profits the Pennsylvania Railroad 'took' from the Erie Canal shipper also allowed a cheaper and more flexible transport of goods for, say, a steel manufacturer in Pittsburg or a salt producer in Syracuse. The bulk of the IT industry is dedicated to making old processes more efficient, and therefore 'taking' profits from some middleman.
The HFT's profits do come from a large mutual fund, but then that large mutual fund can lower it's risk because it can quickly move from one position to another because it can expect a HFT to buy whatever it is selling. Risk has a price (ask an insurance company) and reducing it adds value to the economy.
Perhaps HFT was very lucrative for a while, but competition has naturally squeezed the margins out, but the markets has all the benefits of having HFT around.
And to take your example a step further, the lower risk of a large mutual fund being unable to offload assets quickly enough could result it allocating more funds to an IPO used to raise funds for a company that actually makes stuff.
The caveats are (i) there are diminishing returns to what trades add in terms of wealth because reducing time and cost of offloading stock a further 15 milliseconds and 0.001% clearly isn't going to radically alter the risks of investing in productive enterprises as reducing the time and cost of selling stock by an hour and 5% was
(ii) certain trading behaviours can actually scare real investors away.
Buyers and sellers of stock face a spread anyway. Quants make the spread smaller by arbitraging it. They are now so good at it that the profits are down to 20% from what they were a few years ago. Mutual funds and other investors can now thus buy their stock with smaller spread, i.e. smaller 'salami slice' to give up.
Different trading strategies will make money in different ways. Trading ahead and ticker-tape trading will redirect some of the profits that that an index fund would have made to the HFT trader. On the other hand, market making is in effect providing a service to everyone who wants to trade in the market, and they pay for it (partly through the spread between buy and ask prices -- this is like a fee for being able to buy/sell now rather than later, and partly through explicit ECN fees charged by the exchange).
This is a good question. In financial models, one common approach is the "noise trader" theory where some people make random trades. These are the people who lose out to the informed traders.
As to not creating wealth, it is possible that HFT adds value by seeing where the market is going (slowly) and putting the price there right now. Also, it may be able to extract more information from the truly informed traders, so that the market as able to have more information given less informed traders.
That said, I doubt HFT adds much positive value, but as the rules of the game evolve, ultimately the will favor people who add value.
I think it is an abstraction about how finance is as a whole. I gather you are not in finance, but neither am I.
Let's say I'm a fledgeling lemonade stand and I want $10 of investment. You give me that $10 and with it I make $15. You wanted a 10% return on your investment so you get back $11.
With this model, you have "created" $1 in investments. That $1 originally came from the $15 I made with my lemonade. And all those $15 were exchanges for convenience or sheer happiness gained from having lemonade.
So no one really "lost" in this scenario. But I think the finance people call this "creating wealth". Vs other scenarios, where if you invest in someone who is bad at selling lemonade, you only get $8 back, now you have lost $2 from your initial $10. The idea is to continually "create" wealth by making good choices, investing in good lemonade stands vs bad ones.
So when someone says x-machination (high frequency trading) has "made" $5B, there isn't necessarily a loss anywhere. That's just how much that came out on top.
in your example you left out the person that paid for the lemonade. they lost. i think in an idealistic value system, all value is exchanged not created, a lot like energy in physics. not that i'm saying that's a good model for economics, but it's a valid high level thought, and one i think the original poster might have in mind.
I mean, people 'lose' money all the time for things they get pleasure for. And Even when the thing they gained is very abstract.
Like vegas. People lose money almost by definition.
Or basketball games. Normal people get a -100% return on basketball games. And no one writes articles how the Grizzlies are somehow not providing value to society.
It's the same with capital markets. A lot of people get a huge kick out of speculating on the future. Even if they lose. The capital markets, no matter whether you are an HFT firm in Shoreditch, PIMCO, or some retail investor, are more a game than anything. Everybody wins.
No, it's a tragedy when we start saying that people need to rely on the capital markets. Then, sure, there is loss that's not like basketball games or lemonade. It's a horrible, tragic loss. But then you are talking about policy, not speculation. The real question is why on earth do people rely on the capital markets in the first place? Why is this 'loss' different?
If they can get lemonade cheaper then yes they lost. In this instance there would have been no stand if not for the investor. Chemically speaking the investor is the catalyst that make the whole reaction possible.
Two ways off the top of my head, both mentioned in the article:
Arbitrage: balancing the prices between exchanges. This helps settle all exchanges on the market price, so you don't pay $5 more just because of the exchange you're on.
Market making: By placing both bid and ask orders, algorithms can tighten the bid-ask spread around the price. When the spread is too large, people are disincentivized to trade because neither buyers nor sellers can agree on a common price.
I would guess that day traders lose out, since they miss the opportunity to act on many of these same opportunities. Also I've heard index funds lose several basis points to HFTs when rebalancing.
In theory those posting liquidity will reduce the bid/ask spread, those that are taking liquidity will see a reduction in frictional costs of buying and selling costs. It is the reduction in frictional costs that generates $$$. That and being able to have an edge in whatever orderbook/microstructure you are playing on.
if i want to buy 1000 bitcoins at $121.46 to hang on to for a while, and while i'm buying the price goes up to $121.50 because a HFT is trying to frontrun my purchases, i'm not going to stop buying, its still worth it to me at the higher price. those few pennies, in aggregate across billions of trades, can add up.
If you want to buy 1000 bitcoins then cross the spread, take out the best offer, and buy 1000 bitcoins. If, on the other hand, you want to stick a bid into the market in the hope that someone else will cross and take your (lower) price instead... then yes, other participants will react to that.
Explain in discrete steps how you imagine this works. What was the prevailing offer at the time you placed your order? How many coins did you buy before exhausting displayed liquidity at 121.46? How do you imaging HTFs front-ran your purchase? how would things look differently in the absence of HFT?
That and this sad kicking in the groin of a system that's working through technology issues as it should:
Democrats in Congress would go further. Iowa Senator Tom Harkin and Oregon Representative Peter DeFazio want a .03 percent tax on nearly every trade in nearly every market in the U.S
Politicians just have to get in and start pecking for some extra revenue and attempt to give the illusion that they're going to help.
> During the first week of the tax, the volume of bond trading fell by 85%, even though the tax rate on five-year bonds was only 0.003%. The volume of futures trading fell by 98% and the options trading market disappeared.
Needless to say, 'tax avoidance' became the norm (especially since it was exceedingly easy to circumvent) and it was later repealed in 1991.
In today's deficit boogeyman environment I can't say that extra revenue isn't a consideration, but really the tax was to attempt to stem a runaway financial system that has become the blood-sucking vampire squid[1] that's taking more and more of the wealth while providing little of value to the country.
Like the article points out, the inefficiencies of the market are being removed exactly because of these so called "vampire squid". Maybe we should call them "cleaner shrimp" instead, since they're performing a useful function?
Don't get me wrong, I tried my hand at some day trading a couple of years back and the HFTs helped to make it hard going. That was the problem with the business model I wanted to pursue at the time, though, not with the market itself.
More than that though, the robots only win when they're fighting panicked human traders who try to maximize and fire blind (or at least a few seconds slower). Speaking of intelligent programs, they failed to achieve equilibrium with their natural environment, as Agent Smith would say.
The takeaway:
HFT also lacks the two things it needs the most: trading
volume and price volatility. Compared with the deep,
choppy waters of 2009 and 2010, the stock market is now a
shallow, placid pool.
Yes, that's the integral section of the article, but what it's really saying is that at least one component of this is the current cycle low in volatility and volume. This is a classic wall st shakeout when the best capitalized firms that make it through the down part of the cycle come back much stronger when their market returns because their competition closed up shop. They haven't scared everybody away, they're just suffering a market downturn in volatility and volume right now.
The robots win when people trade more and when prices move around more. When those conditions return, the robot party will be back on.
The second main point: with declining volatility, there is less space for HFT strategies to profit.
What I didn't see in the article was an explicit discussion of transaction costs. Transaction tax would increase those costs, but was part of the decline because transaction costs have already gone up?
So, there's a correlation between HFT and volatility. Yup.
Causation is hard though.
"Trading volumes in U.S. equities are around 6 billion shares a day, roughly where they were in 2006. Volatility, a measure of the extent to which a share’s price jumps around, is about half what it was a few years ago. By seeking out price disparities across assets and exchanges, speed traders ensure that when things do get out of whack, they’re quickly brought back into harmony."
There's been academic research that HFT market making dampens intraday volatility [1]
There have also been findings in the other direction [2].
I suspect to the extent you're talking about market making, the first version is true, but you almost never see it in articles like this. Maybe it makes for poor sensationalism.
Awesome. Capitalism working as intended. There was an inefficiency in the market, people exploited it and low barriers to entry now make the profits disappear. Our stocks are now better priced as a result.
I'm sure it's been done, but I know there is a place for longer term automated trading as well. Computers have an advantage over the super short term, but even on the short, mid, long term, there is money to be made in finding holes based on human behavior over time, which still drives a big chunk of the market.
The implicit claim is that your trading engine won't beat established trading engines and because of volume.
Both can be falsified, by inventing a better trading engine and getting foreign capital into it. Yes the margin for profit isn't that high anymore, but trading engines+trading markets are one card in the stack one can fight with. There is no holistic solution to making maximum profit, except succeeding when others fail. That's why innovators are always rare goods, because capital herds dry it out.
Another key point: "the SEC is sharing information with the FBI to probe manipulative trading practices by some HFT firms" and now has the infrastructure to detect anomalies.
I get why the need to always be one step ahead pushed trading firms to automation. Still, for some reason, this makes me incredibly sad. Maybe it's because I don't fully understand how the world and the economy work. Why does it have to be this way?
We use technology to improve efficiencies in every industry in the world, but when we use technology in the capital markets people freak out. I don't get it.
Huge volumes of money, a perception of it being gambling and playing with numbers rather than productive work, the possibility of screwing over basically the whole of society when it goes wrong, privileged positions for the entrenched traders (people that outsmart the algo-traders get arrested, massive costs to get the millisecond co-lo advantage on HFT etc), total divorce of algo/HFT from the idea of actual investment in companies based on what they do...
There are some reasons that people are suspicious of the entire world of finance. HFT and algorithmic trading really exemplify all of the issues some people have with the money markets in the first place.
It's the co-evolution of parasite and host. These firms are like tape worms, siphoning off nutrients in the hopes the host won't notice. But the host is continually evolving new mechanisms that make it difficult for the parasite to survive. The economy is growing, changing and becoming more efficient, thanks to the internet. If history is any indication, easy-money traders are headed for extinction.
One of its objectives?
Those philanthropic high-frequency traders just wanted to make the market more efficient for everyone? Sure they did. Sure they did.