Going dark: Service disruptions at stock exchanges and brokerages
Are you a bull or a bear? If you can’t access your data and money, do your sentiments about the market still matter?
I was recently asked about how software vulnerabilities in stock trading apps and platforms might put users’ finances and personal data at risk. Given the dependence of today’s societies and economies on technology along with the skyrocketing interest in day trading of late, it’s only natural that concerns about the increasing number and severity of security loopholes in all manner of software applications should rise in lockstep. And that’s on top of numerous other cyberthreats that require the continued attention of organizations and people, including those involved with stock trading.
Recently, a string of disruptions that have plagued stock exchanges and brokerages have thrown into stark relief another problem: an outage, too – even if it’s caused by a technical glitch – can ultimately impact the finances of people and organizations. While this issue typically commands less public attention, incidents that halt trading on platforms where billions of dollars normally move every day may even impact investor confidence and have knock-on effects for countries’ economies. Indeed, I spoke about the importance of ensuring the availability of trading technologies back in 2018; if recent history is any indication, things don’t appear to be improving.
The availability of data and systems is, along with their confidentiality and integrity, one of the pillars of the venerable CIA triad, the concept at the heart of information security and the guiding principle of any organization’s data security efforts. The impact of availability problems varies from industry to industry and from asset to asset; put bluntly, being unable to access a small social media analytics platform is not quite the same as having problems logging into your company’s Enterprise Resource Planning (ERP) application.
Common sense would lead us to assume that the technologies behind stock exchanges are robust, fail-safe, and would never fail under normal circumstances. 2020 proved us wrong – let’s look at how major stock exchanges and brokerages have struggled to keep their systems up and running recently.
Stock exchange blackouts
Tokyo Stock Exchange (TSE)
On Thursday October 1st, the TSE trading session was halted for an entire day. The TSE is the world’s third largest exchange with a market capitalization of about $6 trillion. The outage was attributed to a hardware malfunction in its stock trading system and auto-backup system. Two failures in a row. Nonetheless, the TSE resumed operations on the next day.
This system proved resilient against natural forces, having held up during a powerful earthquake and tsunami in 2011; on the other hand, it wasn’t the first time that its Arrowhead trading system experienced a glitch.
On November 5th, the Japan Exchange Group – the TSE’s owner – announced in a press release that the system had been upgraded. This update offers higher availability and speed.
I ask, were these systems tested regularly, either internally or by the vendor, or was this simply misfortune? Wrong day? Wrong time? Who knows.
Mexican Stock Exchange (BMV)
On October 9th, the trading session at Mexico’s oldest stock market halted at midday due to operational problems with the system used to process trading orders. The stock exchange blamed the outage on a connection cut out mistakenly caused by a technology provider. It’s worth noting that Service Level Agreements (SLAs) play an important role in these kinds of problems.
Even when a technology is resilient and the IT General Controls are audited on a regular basis, people will inadvertently make mistakes. Nonetheless, trading resumed the following Monday with all platforms working normally.
Still in October, trading on several major stock exchanges in Europe also came to a standstill.
Rush hours are at market opening and market closure (09:30-16:00 EST) are the most crucial moments for the market. There is massive buying and selling during these times, with orders being sent to the same API endpoints and the same servers at the same time.
Thousands of users from different brokerages have reported availability problems on their web, mobile, and desktop trading platforms. Angry users were not able to buy or to sell securities at the right price. Millions of dollars vanished in lost opportunities.
In my opinion, regulators should take action against such non-diligent behavior by brokerages.
Retail broker unavailability
After the COVID-19 pandemic caused a huge increase in their user numbers, many retail brokers now suffer from the same problem: availability at opening/closing hours.
Robinhood, one of the most popular platforms, went down in March 2020:
Trading is currently down on Robinhood and we’re investigating the issue. We’re focused on getting back up and running as soon as possible and we’ll update the status page with the latest https://t.co/mON07oWvHy.
— Robinhood Help (@AskRobinhood) March 9, 2020
In December, Robinhood experienced another outage. And so did Interactive Brokers:
We have experienced a significant failure in multiple segments of a highly resilient data storage system. We have engaged the vendor for analysis and recovery. Many services have already been restored and the remaining services are expected to be available promptly. We apologize.
— Interactive Brokers (@IBKR) December 7, 2020
A number of other brokerages also reported availability problems and I’m quite sure many more suffer from this technology “illness.” For example, TD Ameritrade has had multiple availability issues since its merger with Charles Schwab was announced in November 2019. As an end-user of Thinkorswim by TD Ameritrade, I had frequent, unpleasant experiences in 2019 with their mobile and desktop platforms. On some days, I wasn’t able to log in to any of their platforms; on other days, only the desktop application was available for trading. As far as I can tell, the availability problems have been in their authentication and charts servers.
Clients may be experiencing slowness across some of our web and mobile platforms due to heavy volumes. For the best experience, we recommend using the TD Ameritrade web platform at https://t.co/583CCMc1mM, the @thinkorswim mobile app and @thinkorswim desktop.
— TD Ameritrade (@TDAmeritrade) November 9, 2020
The puzzling questions are, since the acquisition announcement in November 2019 and the pandemic worsened in early 2020:
- Was TD Ameritrade ignoring availability complaints just because they knew they would be acquired by Charles Schwab?
- Did IT fail to do some math in terms of scalability to avoid bottleneck issues knowing their userbase increased during the pandemic?
- Did Charles Schwab perform due diligence of TD Ameritrade’s technology?
- Will Charles Schwab invest more in technology going forward to keep their new userbase happy?
Where was Robinhood’s customer service?
Technology and processes don’t work by themselves – both need people to achieve business goals.
So what happened when numerous Robinhood user accounts were looted and there was no one to call? During the first week of October, attackers targeted several Robinhood user accounts and drained their funds. This was achieved through multiple hacking techniques, including by gaining unauthorized access to the email accounts associated with the Robinhood accounts and faking identification to re-enable trading accounts. The victims were left in limbo, since the broker had no emergency or customer service phone number. There was nothing they could do but watch as their money vanished.
Robinhood stated this was not a breach or cyberattack on their end, but of the end-users instead. Some of the affected users contacted the SEC and FINRA, but they declined to comment at the time.
- Security is not only confidentiality and integrity, availability is an Achilles’ heel for financial technologies; if it fails, lots of money can be wiped out.
- Stock exchanges and brokerages are still improving in scalability and resiliency. More investment should be dedicated to these areas.
- Regulators should play a more active role in looking after brokerage availability.
- End-users should stay in touch with their brokerage’s help support to solve any problem during trading hours. They can give guidance on which platforms are available for trading.
Final note: A hypothetical massive DDoS scenario
Now imagine, if under normal circumstances these platforms fail, what would happen if bad actors start sending massive amounts of data to the brokerages’ and stock exchanges’ infrastructure? Are they resilient enough to support these attacks and keep operating normally? We will find out in the next few years.
Thanks for reading!
Alejandro Hernández (@nitr0usmx)
Editor’s note: The views expressed in this article are solely those of the author and do not necessarily reflect the views of WeLiveSecurity.com and ESET.