‘Big data’ is widely discussed within the financial markets, as it is elsewhere; particularly concerning how it could and should be used. Yet there often remains confusion and not a great deal of consensus over what ‘big data’ truly means, and more importantly what businesses can do with it. In the trading world – and specifically within the trading communications space – suchdata can be incredibly valuable.

Within the trading environment, huge amounts of data from multiple channels of communication can be captured, archived and analysed. This includes trade data, blotters and history – everything that takes place around a trade, as well as wider trading floor communications. MiFID II, which is due to come into effect in January 2018, will require all communications that are intended to result in a trade to be recorded and stored. So when we talk about trading floor communications, this means fixed line and mobile calls, emails, and instant messengers.

Capturing the vast array of data passing through an organisation is a tough technological challenge, but doing so is offering firms many competitive advantages. Focusing exclusively on the financial markets, the ability to apply analytics to anything from trade executions and overall trader performance, to finding inefficiencies in a trading workflow, can provide the kind of governance and oversight that was simply not possible in the past.

Storing big data

Take storage, for example. Requirements for data storage are now significant, and maintaining such storage is a job in itself. Storage needs to be resilient and secure, with well-maintained back-ups and the possible use of Write Once, Read Many (WORM) compliant storage solutions – a protocol that guarantees the data is not modified in any way after it is first written to the storage medium in question. In many cases, a lot of data is stored on storage media that, if not already obsolete, is coming to the end of life – such as tapes. Tapes are also a medium that is physically degrading and for which no like for like replacement parts are available.

There are particular risks associated with tapes, and in parallel, increasing pressure from regulators and recognition by banks that those need to be taken care of and pushed to network attached storage. Once this migration has been completed, one of the major benefits – as well as no longer being dependent on old tape-based storage media – is that you can apply new technologies to the historical data. This means that if a regulator makes demands for specific data from five or more years ago, the data can be searched for, accessed and swiftly presented in a way which was simply impractical, if not impossible, beforehand.

As a result of the demands upon financial institutions for secure data storage, the risks and costs associated with storing data on their premises are high. It’s not surprising, therefore, that hosted storage is being considered as an alternative model. For years, some data – such as emails – have been archived in the cloud. Nonetheless there has been resistance to storing other data, such as voice calls, there too. However, we are now at a time when many other things – ultimately all services – couldand should be stored in the cloud. There are clear benefits of making the move: less risk, flexibility and scalability of storage, as well as cutting the cost of investing in large amounts of new infrastructure.

Using big data to your advantage

For compliance, being able to use the available data in the right way when needed is crucial. Demands from regulators can be challenging: firms are expected to be able to respond to any request to produce the right extracts of the vast amount of data that forms massive storage archives. At the moment this can be costly, time-consuming and inaccurate; manually listening for sensitive phrases in phone calls for example, means that the possibility of uncovering anything from breaches of policy all the way through to market abuse is not achievable on a systematic basis.

However, there are now various technologies that offer the possibility of filtering through the hundreds of thousands of recorded communications using advanced phonetic searches and compliance officers can now proactively set up alerts linked to particular scenarios. This means that when certain terms or phrases are mentioned, a workflow is provided that allows you to identify, flag and mark for review any targeted communications. Firms should utilise such technology to test their data capture against specific regulatory scenarios that relate to their own compliance culture, to ensure they can meet regulators’ demands if required to.

Beyond compliance

Beyond compliance, utilising technology to mine the vast amount of data being captured can be a significant benefit to the wider business strategy. From analysing sales if products are being mis-sold or spotting opportunities to cross and up-sell, to monitoring service quality and examining whether staff are following procedure and standards on client calls; the insights gained from big data can direct and drive improvements across business functions and provide a competitive advantage.

As we await the implementation of MiFID II, financial organisations will need to become more proactive in the way they manage and monitor the many channels of communication used every day. Regulations are becoming increasingly stricter and firms need the ability to capture, analyse and store their data as efficiently as possible.

Written by Robert Powell, Global Head of Compliance at IPC Systems