From the safe distance of February 2010, we can look back at 2009, rake over the coals and think carefully about what lies ahead in 2010. Before I start my survey, I must point out that I am indebted to the Financial Information Forum and its membership, especially the exchanges, for providing a vast array of vital statistics and updates. I have data from www.marketdatapeaks.com and my own thoughts of course; but it is not easy to beat the hard cold facts of FIF market capacity data.
Looking back, the first impression is that every exchange has a slightly different story of growth and technological change but in many cases the peaks in December 2009 were double those in 2008 even though December 2009 was a quiet month. In December 2008 we were still feeling the aftershocks of the financial crisis and volatility was through the roof.
The high point of the VIX in December 2008 was 68.51. It was only 23.69 at its highest point in December 2009. Consolidated market data in the U.S. is provided by NASDAQ and SIAC. In particular, CQS carries the national best bid and offer quote data for the listed stock markets. In December 2008, it peaked at 88,249 messages per second (mps). In December 2009, CQS peaked at 166,523 mps.
The peak number for trades per second delivered by CTS in December 2008 was 15,058 and 24,513 mps in December 2009 – a mere 63% higher. As for the consolidated data supplied by NASDAQ in the form of UQDF, the market pumped out 24,817 quote messages per second at the end of 2008. In December 2009 it was 37,086 mps, a 50% increase.
The trades hit 6,167 via UTDF in 2008 and 11,107 mps in December 2009, so that is 80% or so of growth. Not too shabby, you might say! The US equity options market is the largest of all the feeds because there are around 300,000 option prices all being updated at once when the market moves significantly, opens or closes.
In December 2008, OPRA hit a peak of 810,999 mps and in 2009 it made 1,470,242 on December 30. Perhaps it isn’t surprising that the growth in the options market data was fairly similar to the data coming from the underlying markets. There are maybe 400 High Frequency Trading firms and these will all need to bring in proprietary exchange feeds to gain access to market depth and to be able to see quote and trade data faster than is available via the Securities Information Processors.
These proprietary feeds have also been growing at a fair clip in most cases. NASDAQ was the first market to go electronic in the early 1970s and it led the way with SuperMontage in the 1990s. The data feed is now known as TotalView and it carries a very full and active order book. On Exegy, we have seen order books with 13,000 orders on a single side of a single stock.
In December 2008, TotalView made a high mark of 106,963 mps and in December 2009, we saw 253,543 mps. From this we can conclude that the transparent order book data is extremely fast and growing extremely quickly. BATS became an exchange only in August of 2008. By the end of 2008 the FAST PITCH feed was producing 152,719 mps and this became 257,661 in December 2009.
The NYSE Archipelago feed (ArcaBook for Equities) hit 264,469 mps at the tail end of 2008, and was hitting 314,151 mps in December 2009. However, before you utter a sigh of relief that the growth rate wasn’t 100%, I must point out that we might be missing microbursts that last less than a second.
Certainly I have heard several direct feed recipients discussing the problem of measuring the “true” peak. Microseconds matter, especially when you need to configure networks, computers and switching gear to accommodate the bursts. So it seems that the one second time frame for measuring the number of messages is too long these days.
As a result, NYSE Arca and BATS have begun to report data in milliseconds to the FIF. This different reporting time frame sheds a very different light on what networks and hardware have to deal with.
For example, BATS Multicast PITCH feed hit 209,950 mps in December 2009. But this peak was much higher when BATS checked for peaks in 1 millisecond slices of time. On this basis BATS hit 3,408,000 mps on December 22. (This was the peak based on checking for the highest message rate in messages per second across any single millisecond.)
Thanks to ARCA and BATS, the FIF is just beginning to get data on finer increments of time and as the FIF accumulates data we will discover if the microbursts are getting higher or more frequent in 2010. My guess is that we will see more microbursts as all the exchanges are competing fiercely to send out orderbook data as fast as possible and they are therefore constantly tuning and improving systems with this in mind.
Another notable phenomenon of 2009 was increasing use of proprietary feeds for options markets. And not surprisingly, given OPRA’s growth, these feeds grew quickly as well. BONO, DAP and ITTO are feeds from the NASDAQ portfolio. BONO was 60,292 mps, DAP 73,488 mps and ITTO was 73,736 mps in December 2008.
The data rates at the end of last year were, 70,510 mps, 93,182 mps and 94,160 mps, respectively. But these peaks were all measured over 5 second time intervals. NASDAQ started providing data that occurred in one second intervals from May last year and so we know that the more realistic peaks at the end of 2009 were BONO – 98,507 mps, DAP 146,151 mps and ITTO at 147,043 mps.
NYSE Arca Book for Options did not grow much but was at a different level of output in the first place. It hit 682,226 in December 2008 and 735,326 in 2009. We just added this feed to www.MarketDataPeaks.com this January and you have probably noticed that the highest peak is now nearly always over two million messages per second each day.
NYSE Amex Options completely updated its technology and came out of the blocks with its new proprietary options feed in September 2009 with a peak of 684,949. It peaked at 890,244 mps at market open on Christmas Eve. The CME has also been growing. In January 2009, a traditionally busy month, the FIX/FAST feed hit 9,623 mps and in December 2009 it hit 14,868.
Not to be left out, Europe also exhibited strong growth. The FIF only received complete mps data for Deutsche Borse and Eurex from December 2008 and through 2009 but we know that European growth was indicative of a trading revolution involving various alternative trading systems such as CHI-X, BATS, Turquoise, and stalwarts such as the LSE and Euronext.
BATS Europe only started trading in November 2008, and we can now see that it sent 1,066,000 mps on a one millisecond burst on December 23, 2009. So what about 2010? Is it reasonable to assume that we will see roughly double the peaks of 2009 in 2010? Yes! It is safe to assume that a doubling of data is a good starting point as a rule of thumb.
The drivers of regulations, competition and technology are all in place. So if you are planning on low market data growth because the VIX volatility index has diminished, then you are probably taking a large risk. It might be possible to save money in the short term by guessing that the market data growth will ease, but at any moment a microsecond burst or series of sustained bursts could render market data systems useless and knock trading systems out of commission just when they are most needed.
If you are serious about planning, you should look carefully at the exact data sets that you are bringing in. Not all exchanges are the same. And some of them like OPRA do issue very clear guidelines well in advance. Some people are pointing out that the regulations are a wild card this year since they are under review both in the U.S. and in Europe.
Also there is talk of bringing High Frequency Trading under government scrutiny. The SEC is looking co-location services as part of its Concept Release Review of Equity Market Structure. But my guess is, even if the regulators ban proprietary trading by big broker dealer banks, it is extremely unlikely that they stop all banks and High Frequency Trading firms from operating.
So you certainly shouldn’t be planning for amelioration in market data growth rates in 2010. In fact, the U.S. stock exchanges are now lobbying for the regulators to contemplate sub penny increments. This could be another spur for market data growth since price quotes would have to go through so many more price levels just to move a penny.
There is also a risk that the regulators will take an increasingly dim view of dark pools and force more trading onto the exchanges. Again, this would simply add to market data growth. Another reason to anticipate growth is the introduction of new markets such as PSX from NASDAQ in May 2010 and the launch of BATS Y.
We also expect to see the accession of Direct Edge to exchange status with two exchange licenses A and X and that could lead to yet more volume. In the realm of equity options, if you imagine for a moment that the existing options markets have topped out, you have to counterbalance that assumption with the fact that we are expecting two new exchanges this year-
BATS and C2. BATS will go live on February 26 and C2 is expected to start up in August. OPRA has planned for plenty of growth based on the following drivers:
* increased competition
* the expansion of penny trading
* increased volatility in the underlying
* new products
* industry-wide growth in options trading
* potential market model changes
* increased quoting participants in the markets.
With this in mind, OPRA has set the expectation that the new ceiling in market data for the OPRA feed will be 3,450,000 mps or 590 Mbps from July 13th. And most recipients would expect to provide for double this bandwidth in order to receive redundant streams.
If you are thinking of databasing all that, it could be 10.3 billion messages in total over the course of a day. So capacity will continue to be a headache for many firms in 2010 but at least you can plan with data available from the FIF and monitor data rates in realtime on www.marketdatapeaks.com.
By the way, FIF members will be able to login to MarketDataPeaks later this year to see the peaks for the different markets that comprise the aggregate message rate. FIF members can also call into the monthly meetings to listen to advice from the exchanges first hand. Notably, the LSE has recently joined and they expect to provide data, so this time next year we’ll have a much more complete picture for both for sides of the Atlantic.
Subscribe to our newsletter
