# Data integrity: the holy grail of operational efficiency
Author:  Pal Sinha, Barnali 
Author URL: https://financedigest.com/author/pal-sinha-barnali
Published: 2020-11-09
Category: TECHNOLOGY
Category URL: https://financedigest.com/category/technology
Meta Title: Boosting Resilience: The Holy Grail of Operational
Meta Description: Discover how businesses are adapting to the challenges of the pandemic by improving data accuracy, streamlining processes, and embracing automation for greater
URL: https://financedigest.com/data-integrity-the-holy-grail-of-operational-efficiencyhtml

![undefined](https://prod.superblogcdn.com/site_cuid_cm5qst7v3003gwirgwqtxn8i8/images/untitled-design-2020-11-09t103606-1736838739794-compressed.jpg)

_By **Douglas Greenwell,** Director, Duco_

While those in the field of medicine are chasing a vaccine for Covid-19, firms elsewhere are seeking a different holy grail: operational efficiency in extraordinary times.

The pandemic has created an unprecedented environment for businesses, forced to adapt overnight to remote working and accelerate their pace of digitalisation. A recent report from the International Data Corporation predicts firms will invest $6.8 trillion in digital transformation between now and 2023 [\[1\]](#_ftn1). Flexible, agile [businesses have been able to operate far more easily in current conditions than those still reliant on manual tasks and outdated technology](https://www.financedigest.com/extracts-and-distillates-market-overview-high-demand-recent-trends-future-growth-business-scenario-technology-2022-2029.html "Extracts and Distillates Market Overview, High Demand, Recent Trends, Future Growth, Business Scenario & Technology 2022-2029").

A major pain point has been exposed in [finance and operations](https://www.financedigest.com/stellantis-reshuffles-european-financing-operations-through-new-jvs-with-banks.html "Stellantis reshuffles European financing operations through new JVs with banks"), where spreadsheets and legacy technology are far more prevalent than in customer-facing systems and front-office processes. According to Aite Group, 47% of firms are still processing data manually in these functions [\[2\]](#_ftn2).

As we enter a winter of [uncertainty faced with economic](https://www.financedigest.com/combatting-economic-uncertainty-in-2023-with-the-payments-sector.html "Combatting economic uncertainty in 2023 with the payments sector") turbulence, the need to tighten resilience, boost efficiency and reduce costs through improved data accuracy, slicker processes and improved automation has never been greater.

**A decade of data**

We are living in the ‘decade of data’, with [businesses dealing with trillions of lines of data created from a multitude of sources every day](https://www.financedigest.com/nz-aussie-dlrs-have-busy-day-fed-minutes-loom.html "NZ, Aussie dlrs have busy day, Fed minutes loom"). While most [firms will have legacy systems](https://www.financedigest.com/prefabricated-building-system-markets-advanced-research-growth-opportunities-in-global-industry-by-2025-edited-by-leading-research-firm.html "Prefabricated Building System Market’s advanced Research & Growth Opportunities in Global Industry by 2025 Edited by Leading Research Firm") in place to deal with a large percentage of this data, anything complex or bespoke will usually need some form of manual processing.

The recent track and trace database disaster was a timely reminder of the risk of depending on Excel, with its wide margins for [error and unsuitability for reliably processing data](https://www.financedigest.com/outthink-raises-10-million-to-tackle-human-errors-behind-data-breaches.html "OutThink raises million to tackle human errors behind data breaches") at volume. Accounting teams typically [spend time](https://www.financedigest.com/its-time-for-banks-to-spend-on-digital-mortgage-journeys.html "It’s time for banks to spend on digital mortgage journeys") waiting for complex calculations to run across tens of worksheets, often using inherited processes built over years by multiple owners.

The expected [trajectory of data explosion means that continuing](https://www.financedigest.com/multilocal-continues-hiring-spree-as-its-steep-growth-trajectory-continues.html "multilocal continues hiring spree as its steep growth trajectory continues") with such high risk and labour-intensive processes is simply not sustainable. Switching to a more intelligent [way of working](https://www.financedigest.com/five-ways-to-embrace-hybrid-working.html "Five Ways to Embrace Hybrid-Working") is a question of ‘when’, not ‘if’. The exceptional circumstances presented by the pandemic has created a stepchange in mindset – automation [needs to happen at a far greater rate](https://www.financedigest.com/boes-pill-sees-need-for-further-interest-rate-rises.html "BoE’s Pill sees need for further interest rate rises") than ever before.

**Boosting resilience**

The pressure is on finance and [operations teams to modernise middle- and back-office processes and boost both resilience and business continuity through better data](https://www.financedigest.com/libor-reform-should-trigger-a-major-reassessment-of-corporate-treasuries-data-operations.html "LIBOR reform should trigger a major reassessment of corporate treasuries’ data operations") management. Firms are actively seeking [ways to make complex data easier to manage and transform](https://www.financedigest.com/four-ways-data-and-ai-can-transform-financial-services.html "Four ways data and AI can transform Financial Services") systems to ensure data accuracy and a single data truth.

Reconciliation is an important part of this process and an area that we at Duco have seen [firms focus on in the past](https://www.financedigest.com/aussie-dollar-firm-as-bonds-tumble-yen-looks-past-boj.html "Aussie dollar firm as bonds tumble, yen looks past BOJ") six months. Earlier this year we developed a [Reconciliation Maturity Model](https://content.du.co/reconciliation-maturity-model-whitepaper) that identifies a five-step journey organisations can take as they progress from manual reconciliation, through to part-automated, fully-automated and ultimately self-optimising systems that use machine learning to ensure optimum reconciliation efficiency.

The [majority of firms](https://www.financedigest.com/bnp-paribas-buys-majority-stake-in-dutch-firm-dynamic-credit-group.html "BNP Paribas buys majority stake in Dutch firm Dynamic Credit Group") sit somewhere in the middle, often in a ‘hybrid’ stage with inflexible technology able to automate some of the simpler tasks but unable to handle anything complex or bespoke. This [creates a patchwork of fragmented processes](https://www.financedigest.com/growth-of-mineral-processing-and-mining-industrie-is-creating-demand-for-rotary-trommel-market.html "Growth Of Mineral Processing And Mining Industrie Is Creating Demand For Rotary Trommel Market") and can result in inconsistencies, errors and duplicate work.

We have found that Covid-19 has accelerated the speed at which organisations have wanted to [move beyond](https://www.financedigest.com/the-high-fiber-feed-market-to-move-beyond-the-monotony.html "The High Fiber Feed Market To Move Beyond The Monotony") this – often far faster than the pace that change programmes usually progress at. [Firms are now actively seeking ways to make reconciliation easier to manage](https://www.financedigest.com/asset-management-firms-are-using-ai-to-deliver-higher-returns-for-investors.html "Asset management firms are using AI to deliver higher returns for investors") through modern, flexible systems. The kind that can handle any format of [data and be relied upon to solve multiple](https://www.financedigest.com/how-to-navigate-multiple-data-privacy-regulatory-frameworks.html "How to navigate multiple data privacy regulatory frameworks") issues across all roles and functions.

In the past this could have been an arduous process, but [businesses now seem much more comfortable with moving reconciliation systems to the cloud](https://www.financedigest.com/how-cloud-computing-saves-businesses-money.html "How cloud computing saves businesses money"). These systems are available on demand (in some cases, just 24 hours), enabling operations and [finance teams](https://www.financedigest.com/protecting-the-finance-team-from-burnout-to-drive-employee-retention.html "Protecting the finance team from burnout to drive employee retention") to set up processes far quicker than on legacy platforms. Cloud-based systems are also far easier to maintain with a remote, distributed workforce.

**Maturity through [machine learning](https://www.financedigest.com/transforming-insurance-through-artificial-intelligence-and-machine-learning.html "Transforming Insurance Through Artificial Intelligence and Machine Learning")**

Once such modern systems are in place, firms can start to [benefit further from having clean digitised data](https://www.financedigest.com/the-benefits-of-big-data-how-to-convince-your-cfo.html "THE BENEFITS OF BIG DATA – HOW TO CONVINCE YOUR CFO") that they can trust. Machine learning technology, for example, is not particularly [effective if it is fed poor quality training data](https://www.financedigest.com/can-banks-compete-in-the-digitisation-race-without-effective-data-integrity.html "Can banks compete in the digitisation race without effective data integrity?"), or if a significant amount of that data is still being processed manually.

With a modern, machine learning-enabled reconciliation system in place, organisations can greatly reduce reconciliation setup and investigation times. The [system will be able](https://www.financedigest.com/emerging-risks-is-our-insurance-system-able-to-cope.html "Emerging risks: is our Insurance system able to cope?") to analyse the data inputs, suggest matching rules, then label and triage any exceptions – all based on past results.

Cloud-based systems have another advantage here. While machine learning [technology can be deployed on an on-premise system](https://www.financedigest.com/sensor-based-glucose-measuring-systems-market-by-2028-with-technological-advancements-growth-of-industry-overview-and-dynamics-with-affecting-factors-2.html "Sensor Based Glucose Measuring Systems Market by 2028 With Technological Advancements, Growth Of Industry, Overview and Dynamics with Affecting Factors"), the amount of training data is limited to just that one organisation. However, a cloud-based [system can learn from industry-wide data](https://www.financedigest.com/interoperability-of-data-to-accelerate-the-ventilator-test-systems-market.html "Interoperability of data to accelerate the Ventilator Test Systems Market") and make suggestions based on best practices. This is similar to a traffic app giving you suggestions based on all the journeys its users have made, rather than just the journeys you have made yourself.

As outlined in the Reconciliation Maturity Model, the end goal for reconciliation is a self-optimising model, where every part of the process is [automated and the system](https://www.financedigest.com/test-automation-systems-for-powertrain-dynamometer-market-to-reach-worth-of-us-1-20-bn-by-2031-tmr-insights.html "Test Automation Systems for Powertrain Dynamometer Market to reach worth of US$ 1.20 Bn by 2031, TMR Insights") is able spot data quality issues at source. This removes the need for internal reconciliation, leads to significant cost reductions and frees up teams to spend their time on [higher value](https://www.financedigest.com/delivery-hero-sees-higher-gross-merchandise-value-in-third-quarter.html "Delivery Hero sees higher gross merchandise value in third quarter") tasks and projects rather than inefficient processes.

While the [technology required for this utopian future](https://www.financedigest.com/nutragenomics-market-overview-forthcoming-technologies-future-trends-latest-insights-by-share-outlook-till-2022-2029.html "Nutragenomics Market Overview, Forthcoming Technologies, Future Trends, Latest Insights By Share, Outlook Till 2022-2029") is not quite ready yet, firms are starting to recognise the steps needed to get there. The transition to machine learning may not happen overnight, but the shift away from spreadsheets to flexible, cloud-based software is taking place faster than ever before.

[\[1\]](#_ftnref1) https://www.idc.com/getdoc.jsp?containerId=prUS46967420

[\[2\]](#_ftnref2) https://aitegroup.com/report/trends-reconciliation-technology-ai-trained-recs-or-train-wrecks


---
This blog is powered by Superblog. Visit https://superblog.ai to know more.
---