Continuous Monitoring– Is your organization Ready?
All organizations today have gone for significant level of Automation for their business transaction processing. Also, there is growing realization that as the transactions are on-line, there is need to shorten the Audit cycle, increase sample size and move towards real-time monitoring. Auditors typically define key processes that need to be monitored with pre-defined frequency and parameters and exception criteria that are measured closer to real time basis to improve effectiveness of Audit. The success of continuous monitoring solution differs from industry to industry and organization to organization. In order to have effective deployment of continuous monitoring solution, one has to assess the extent to which an organization is ready to move in the same direction. Having shared vision, assessing process maturity levels, understanding process variances, identifying key measurement parameters and collaborative approach are the parameters to determine if the organization is ready for continuous monitoring. Let us quickly understand the same in more details. Shared Vision Continuous monitoring projects have long term investments and long term impact on Audit and monitoring function. It is therefore necessary that the Audit team itself is ready to invest resources in the project. Shared Vision Considerations Ability to visualize This looks obvious. From Auditor perspective this is a complete process re-engineering and increased thrust on back-end analysis and increasing sample testing, possibly across regions and geographies. Auditor should be able visualize process change within feasible and practical parameters. Ability to gauge complexity The process of implementation of continuous monitoring requires deeper understanding of all processes, correlations, data mapping, and exception identification. In a diversified environment this complexity could be more than expected. Time commitment The Audit team needs to have belief and buy-in the solution proposed. The monitoring projects require commitment of time and there should be willingness to invest in the same. Long term sense with milestones Continuous monitoring projects are long term in nature and therefore needs suitable milestone based project plan. Process maturity and process variance It is also important to understand process maturity of the organization. Organizations do have process gaps in all areas across various phases. However, in order to monitor such gaps through the system, it is necessary that the base system should at least have a resemblance of stability. It is generally recommended that the processes that are at least “defined” (as per CMM model level 3) should be taken for any monitoring purpose. Process Maturity Considerations Diversified data sources In a diversified organization, there is a possibility that for measurement of same process parameters, there are different data sources themselves. For example, when you are analysing invoice data, there could be different invoice formats for different types of customers. This needs to be factored in effort estimates. Reliability of sources is more important than ETL Typically, many companies operate in partially automated environment. There exist peripheral systems, end-user developed interfaces, manually compiled data. From Audit point of view, this data is critical to be considered for analysis. More than efforts for ETL, Auditors need to verify if the sources are reliable, consistent and usable. KPI identification. The KPI development process requires huge investment in terms of time and money. Processes that are routinely monitored by other tools and techniques or have less impact on the organization may not require investments. KPI identification Considerations Is KPI Critical Auditors need to apply the criteria to ensure that they focus on important parameters that will yield better results. For example, in a travel business, a pattern of tour cancellations and refunds may need closer monitoring compared to x Is KPI Unique Usually, Auditors have query access to many base systems and an exception can be monitored through system generated reports. Through experience Auditors sense an outlier or a red flag that is either not available in base system or Auditors may like to have an independent view of the exception. What are KPI Parameters The KPI has to be understood in holistic perspective. Auditor will need to consider filters based on dates, transaction types, geographical locations, projects/ cost centre / profit centre. Auditors will also need to define what constitutes “red flag”, that is the deviation beyond which Auditors will need to scrutinize manually. Level of Automation desired. Audit team can decide if they desire to have fully automated exception reporting. Generating red flags with an acceptable tolerance may not be possible through full automation. Auditors need to decide alerts required on real-time basis or on monthly basis or by intervening into the system. Useful life Useful life any KPI my change if – An organization is replacing an existing system. – A new process is being introduced The useful life is important to ensure that the benefits of the efforts get realised in the same period. Variances in process measurement. The ETL (extraction, transformation and load) process is considered to be most important to ensure sanity of input data. However, sometimes, some of the processes of the organization could have wide variances across. These variances need to be understood before estimating efforts. Variances in Processes Considerations Format variances Organizations do have shadow IT systems for various reasons. There is likely that the formats of data generated through the system could vary from unit to unit or even from period to period. Missing Data links It is possible sometimes in the organization that entire data link is missing. This could be due to non-existence of automation, loss of data or incomplete capturing of details. Field variances Variances are possible in interpreting fields as well. The fields to watch out could include time zone, location, effective/ posting dates, currencies, number of decimals, transaction status. Incorrect interpretation of fields can lead to re-work and over-heads. Collaborative Approach Audit teams need to decide what should be the approach towards continuous monitoring. Whatever may the approach, there needs to have collaborative effort. Approach Consideration Turn Key Project This approach is useful when resources and funds are available, the organization is “ready” as mentioned above and organization expects faster ROI Specific Objective based development This approach is useful when organization has reasonable understanding of critical areas and would like unfold the development with specific milestones Conclusion Technology tools, navigation through the processes, ETL, data mapping, KPI and measurements are essential components to deploy continuous monitoring solution. Each component needs deep understanding, greater skill and intense effort. However, the degree of success of continuous monitoring depends on other softer dimension as well. The overall organization needs to be “ready” to deploy such solutions. Incomplete understanding of the same may result in underestimation of efforts, time and cost overruns. A proper alignment of shared vision, detailed approach and collaborative efforts are key success factors to ensure the success of continuous monitoring.
Bol Madhavache 