Using Spreadsheets for SOX Documentation: Pros, Cons, and Hidden Dangers
By Tim J. Leech, FCA-CIA, CCSA, CFE
July 2005It is important that I declare for the record that I have a conflict of interest. I work for Paisley Consulting, the dominant SOX software vendor in the market. Writing this article is somewhat akin to a tractor salesman in the early part of the 20th century promoting the benefits of tractors over horses and mules for ploughing the fields. Tractors at the time were far from perfect but it became clear early on that tractors were capable of significantly outperforming horses and mules. The problem was that many farmers were very accustomed to doing their work this way and sometimes were even emotionally attached to their horses and mules.
This article attempts to convince readers that specialized SOX/ERM software offers major benefits over spreadsheets for SOX, while recognizing that many people are accustomed to and like using spreadsheets for every possible application – whether it makes economic sense or not.
History of SOX
The Sarbanes-Oxley Act of 2002 ("SOX") was enacted in record time in the summer of 2002 in response to a "perfect storm" of corporate corruption and failures. By far the most onerous element of SOX, and the one that has drawn the most vocal criticism to date, has been Sections 302 and 404. These two sections force companies to formally document and verify the support for their representations related to control effectiveness and deficiency reporting. The task of completing the first round of SOX documentation has been the biggest contributor to the billions of dollars companies have spent on SOX to date. Maintaining SOX control assessment documentation is expected to continue to be a major expense for companies in the years ahead. A key question companies should be asking is: How can we cost effectively document and store the necessary SOX control assessment work, meet all the technical SOX/SEC/PCAOB requirements, and still gain maximum business benefits? The answer is: Not with spreadsheets. Which SOX software you pick for the job is a more complicated decision.
Using Spreadsheets for SOX -- the PROS
Surveys indicate that the majority of companies impacted by SOX have elected so far to tackle SOX using a combination of word processing and spreadsheets - the "low tech solution". Reasons advanced for going this route include:
Using Spreadsheets for SOX -- the CONS
Difficult to show users the link between accounts, risks and controls -- the logic-based approach
If having controls that actually do what is required is an objective, it is essential that the people responsible, and those that oversee them, are able to see the relationship between objectives, risks, the controls in place, and, ideally, the actual results being achieved. This fundamental continuous improvement prerequisite underpins the total quality movement and well known process analysis techniques like Six Sigma. Spreadsheets are not well equipped to display and continuously track that relationship.
Don't foster work unit ownership and accountability
A key feature in many of the SOX software products on the market is the ability to assign "owners" of "sponsors" to control representations, processes, deficiency disclosures, control testing, verification of control testing, and other important elements. This functionality allows companies to efficiently "divide and conquer" by electronically assigning accountability for the many activities that must be done to support a SOX representation. While a very diluted form of accountability and role definition is possible using spreadsheets alone, it is not easy to accomplish.
Poor security and reliability -- PCAOB
Audit Standard No. 2 requires that an IT general controls assessment be completed on all applications used to assess and monitor controls -- including spreadsheet applications used for SOX assessments.
AS#2 Paragraph 40 states that the external auditor when assessing management's assessment process must evaluate:
AS#2 Paragraph 53 states the assessment must include company level controls including:
A CFO magazine survey published in March 2003 reported that only 11% of 245 CFOs said spreadsheet-based control reporting was accurate enough to make senior executives confident about certifying their companies' financial statement data for SOX. Although supplemental software packages are emerging to try and improve the IT general controls over spreadsheets, it is still a very difficult task to persuasively demonstrate security over unauthorized changes and appropriate version control. Many companies are avoiding the task of assessing the IT general controls over their SOX assessment spreadsheets.
Expensive to maintain over the longer term
Spreadsheets, on the surface at least, appear to be a very inexpensive option for SOX assessment work. Most companies and their auditors and advisors already have enterprise level licenses because of the global dominance of Microsoft Office. The savings is more illusory than real. In round one, because of the time urgency, few companies tracked the full range of cost drivers including the time consumed of internal staff, the cost of any external contract staff, and the time charged by the company's external auditor. Once companies begin to address ongoing SOX costs, including the section 302 requirements to report on material changes in the control environment, provide updates on progress resolving significant deficiencies and material weaknesses, and on a quarterly basis report new significant deficiencies and material weaknesses detected to the audit committee and external auditor, the real costs and deficiencies of using spreadsheets for SOX documentation will begin to emerge. Fullfeatured SOX software packages are available on the market starting at $1,000/user.
Don't allow management or auditors to see the big picture
An important requirement that many companies have not given much attention to in the first round of SOX is the requirement that the entire universe of accounting control deficiencies be analyzed to determine if, collectively, there is a pattern that requires escalating a number of non-reportable control deficiencies to significant deficiency status or material weakness rating. This needs to include analysis of patterns by COSO control
When SOX assessment work is contained in what may be hundreds, or even thousands, of individual spreadsheets the ability to see patterns by COSO category, by process, by account or note disclosure is difficult if not impossible. Research done by the author as part of an analysis of an FEI Research Foundation study on SOX deficiency reporting indicates that companies and their external audit firms have not been consistent in how much attention they pay to this SEC/PCAOB requirement. It is expected that the attention paid to this requirement by external auditors will escalate once the PCAOB inspectors begin to sanction audit firms that ignore it. (It is important to note that many external audit firms also use spreadsheets as the foundation for their section 404 work and they will also have great difficulty spotting patterns that collectively indicate a reportable control deficiency.)
Don't build control assessment skills
Because of the time urgency and the heavy workload in round one many companies used outside consultants and/or project teams to complete the first full set of risk and control assessment documentation. This was often done using pre-populated assessment templates provided by the consultants. Although this approach got the job done it missed an opportunity -- the opportunity to introduce the fundamentals of risk assessment to staff all across the company. Spreadsheets, especially those with "canned" questions, do not allow users to see the one to many relationships between specific risks, the controls in use and the resulting residual risk position and they don't foster work unit ownership and maintenance of the assessments. SOX can be used as training tool to teach a logic-based approach to risk assessment to lay a foundation to "embed" risk and control assessment across the organization.
Not useful for identifying unnecessary controls
Many companies are now realizing that SOX provides an opportunity to examine business processes and identify controls that provide very little incremental value. This step is best achieved by using an approach that identifies the relevant account "assertions", (e.g. existence, valuation, cut-off, etc) or stated another way the "assertion risks" (i.e. real or potential situations that would cause the account or note to be wrong) and then identifying the key controls in place to mitigate those risks. Once this step has been completed controls currently in use that are expensive and/or provide limited incremental assurance related to the assertions can be examined for possible elimination. Once employees fully understand the key steps in the core risk and control assessment process it is a natural extension to engrain the habit of questioning why any control or procedure is being done and whether it is really necessary.
Senior management can't easily prove they are actively overseeing the SOX assessment process
Section 302 requires that management report any significant deficiencies and material weaknesses that are detected during the year to the audit committee and external auditor on a quarterly basis. There must also be a process in place to allow the CEO and CFO to identify and report any material changes in the control environment, both positive and negative, in the quarterly filings to the SEC. Spreadsheets are not well suited to real time monitoring of control status and demonstrating that senior executive are actively overseeing the process that supports the representations they sign. The "Bernie Ebbers" defence, "I didn't know", isn't likely to be successful.
Don't lay a foundation for ERM and the related benefits.
There is rapidly growing body of research that suggests that companies with effective ERM systems are, all other things equal, likely to outperform their peers. Spreadsheets are not optimal when the goal is to continuously identify new risks, significant changes in risks, identify dangerous residual risk situations or "embed" the core skills necessary to create "risk aware/risk responsive"organizations. Although SOX relates only to the reliability of external accounting disclosures, if the assessment work is done using an ERM approach to assessment it can be easily extended to cover the wider array of risks that companies face.
Don't allow efficient integration of testing done by management and internal and external auditors
Once a company has completed the first full round of SOX control design assessment documentation the next step is to test that the "key controls" are in fact resulting in an acceptable error/exception rate. This testing work should be done in a way that CEOs and CFOs are able to see how much work has been done to support the organization's assessment and the representations they are personally making to the SEC in 10K and 10Q filings. The use of spreadsheets does not allow for effective integration and secure storage of testing work on the control assessment documentation. It also does not keep the costs of planning, scheduling and performing the required testing to the minimum amount possible.
Difficult to archive to support quarterly and annual CEO/CFO representations
Under the rules established by the PCAOB external auditors must maintain 7 years of history of the work they performed to audit and report on management's assessment of control effectiveness. Although the SEC does not state a specific record retention requirement to support section 302 and section 404 representations, it is recommended that companies retain the same amount of history. Although this is technically possible to do with a large collection of spreadsheets, it is difficult to do in a way that clearly demonstrates the specific support that existed for the control status representation being made by the company's CEO and CFO each quarter because the documentation does not "sum" to a concise view or report to support the representations being made, including any exceptions.
The data must eventually be moved from spreadsheets to a SOX software platform and spreadsheets hide major technical deficiencies
Some companies on the advice of consultants and/or their external auditor elect to go with spreadsheets for the "SOX foundation year". When the deficiencies of the spreadsheet approach to SOX are recognized in subsequent rounds companies realize they must upgrade to a more robust platform. This requires importing the data stored on potentially hundreds or even thousands of spreadsheets in to the SOX software product selected. The more technical deficiencies there are in the spreadsheet data, the more costly the exercise.
Common deficiencies we see in practice with spreadsheet solutions include not linking process assessment documentation directly to specific accounts and/or note disclosures, not identifying relevant "assertions" for each account and note disclosure, not identifying and assessing the key risks that threaten the reliability of the account or note disclosure, not showing the link between the risks identified and the processes and controls in use to mitigate them, not including information on account transaction volume, dollar activity, and importance of the account to industry analysts and debt covenants, not linking controls identified to the COSO framework, not completing the mandatory macro level anti-fraud assessment, not completing the mandatory macro assessment against COSO criteria, not completing the necessary IT general controls work, not analyzing the population of control deficiencies to determine if collectively they constitute a reportable control deficiency, not analyzing the control environment each quarter for material changes, not providing the mandatory disclosures each quarter on progress to resolve reportable control deficiencies, and others.
Is it Time to Retire your Trusty Spreadsheet Approach?
Spreadsheets are widely available and many people feel very comfortable using them. Unfortunately, for SOX applications, this is a route that comes with many hidden dangers and missed opportunities. Just as tractor salesman in the early part of the 20th century had difficulty convincing some farmers to switch from horses and mules to farm tractors, it is an equally challenging job to convince people that are accustomed to and like using spreadsheets to make the move to SOX assessment and documentation software. I think it all comes down to a simple question. How much yield you want to get from your SOX efforts and spending?
TIM LEECH is Principal Consultant and Chief Methodology Officer at Paisley Consulting, the Cokato, Minnesota-based business accountability solutions software company. He can be reached at Tim.Leech@paisleyconsulting.com2005 Paisley Consulting. Reprinted with permission.