Automatic Aggregation in Auditing - Rutgers University

19th World Continuous Auditing and Reporting Symposium Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Philip Elsas ComputationalAuditing.com Newark, New Jersey November 6-7, 2009 Introduction 1 Since 2003: Company - Canada, Netherlands Offering software and consultancy services to innovate audit practices and audit software firms 1988-2003: Deloitte. with intermezzo at Bakkenist Management Consultants, sold to Deloitte. - Principal, chief architect & inventor of Smart Audit Support - Smart Audit Support: since 1994 key in Deloittes worldwide audit practice. Currently integrated in The Deloitte Audit - System blueprint in chapter 5 of 1990-1996: PhD Computational Auditing - PhD in Mathematics & Computing Science, on Financial Auditing - In parallel to Smart Audit project, 30% part-time, Vrije Universiteit - Directly after appearance awarded with the biennial Alfred Coini Prize for the best publication in Auditing The Dutch Tax Office used Computational Auditing in 2001-2003 as frame of reference to compare Big 4 planning and decision-support models & systems to investigate how to improve audit productivity (57 page report); considers Smart Audit Support leader of the pack ComputationalAuditing.com Agenda Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: What is the content?

Web platform for audit support: How to use that content? Managing the use of aggregation & classification Aggregation mechanisms: quantitative, qualitative & confidence Royal NIVRA: Golden opportunity for the audit profession, Identify a way to contribute to systemic risk anticipation ComputationalAuditing.com 2 Web platform for audit support: Whats the content? by auditors, for auditors 3 ACL AuditExchange (AX 2), Business Assurance Platform Audit repository: data, scripts for analytics (CM), findings CaseWare Open Engagement & CaseWare IDEA Working paper templates & scripts, DMS & KMS, partially organized per type of industry (website building system) Audit support architecture of a big audit firm, or of a shared back-office of a group of smaller audit firms Platform of audit packs* with check lists & audit planning templates,organized per type of industry Deloittes Builder Player Platform-architecture All mentioned + capturing context to offer guidance in determining & configuring scripts for data analysis, addressing the key questions: Interactive - When to do which test? Audit - What to do with the test results? * Audit pack: a bundle of interrelated forms, specific for an industry, or sector Documentation

ComputationalAuditing.com Correctness by Construction p.334 Proven Architecture 4 Deloittes Smart Audit Support: Interactive Audit Documentation published in Word and browsers, Worlds Strongest Audit Support* Conditional Relevancy Flexible Questionnaire integrated in Web Forms: By making explicit what is needed to answer When to do which audit test? & What to do with the test results? you articulate a body of multiple-choice questions, tables, etc., connected by choicelabeled relevancy links, embodying an approach, a method, or even, if possible, a workflow process, to guide how to achieve assurance Effective: dont miss relevant issue Specified Audit Methods drive integral Planning, Execution & Documentation Drives & Captures the Story of the Audit Adequate Instantaneous Efficient: no access to less relevant issues ComputationalAuditing.com p.337 Optimal mitigation of litigation risk * Dutch Tax Office 5

Deloittes approach Smart Audit Supports document index related to Deloittes International Audit Approach PERFORM PREENGAGEMENT ACTIVITIES PERFORM PRELIMINARY PLANNING (1990s) Example audit pack Assess Engagement Risk Establish Terms of Engagement Understand the Client's Business Understand the Accounting Process Perform Preliminary Analytical Procedures Determine Planning Materiality Develop Client-Service Objectives ASSESS RISK Understand the Control Environment Assess Risk at the Account and Potential-Error Level Yearly ROI guess: 20K man-yrs/yr x $10K cost reduction/man-yr = Specific Identified Risk ? $200MNO Rely on Controls YES DEVELOP AUDIT PLAN

No Specific Identified Risk Control Reliance Strategy ? YES NO Identify Controls and, Identify Controls if Efficient, Establish That Mitigate Risk a Rotation Plan In addition to $200M yearly cost reduction ROI is: - Relevant Doc & Planning, no more no less - Comfortable & stringent way to get it Test Controls PERFORM Perform Focused Perform Basic Level Perform Intermediate AUDIT Substantive Tests of Substantive Tests Level of PLAN Substantive Tests Evaluate Results of Tests Perform Financial Statement Review Perform Subsequent Events Review All docs are Allplanning planning docs smart forms with built-in are smart forms Conditional Relevancy CONCLUDE AND REPORT p.336 ComputationalAuditing.com

Obtain Management Representations Report on Financial Statements and Render Management Letter p.62 6 Interactive Audit Documentation: Dedicated Functionalities for the Audit Team Filling out a web-based questionnaire with multiple-choice questions: Documents and guides: Functionalities for audit workflow operators What has been done? & What has to be done? What information has been found? & Whats the impact on the audit? Activates relevant, more detailed questions & de-activates irrelevant Aggregates audit risk/audit evidence, according to a prescribed processing scheme, as captured in risk summarization tables Plans and configures audit tasks to constitute an audit plan, for example, based on accumulated risk: To be able to rely on a specific assertion level control To further investigate the risk by planning substantive procedures Shows when to stop investigating an account, a process or an assertion

Sets a risk classification to significant inherent risk Activates dedicated support to indicate how to: Specify a norm for an entity-level control Specify a fraud risk, including a description of who is able & how to do it Specify a norm for initial numerical analysis; when within norm, no extra tasks Specify or configure a script for a data analysis tool Decide to involve an external specialist in your audit team (e.g. forensic, EDP) Capturing the Story of the Audit, ISA 315.122 ComputationalAuditing.com The Auditors New Clothes, 2008, Tom Koning & the Audit Navigator, translation into English is pending Agenda 7 Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: What is the content? Web platform for audit support: How to use that content? Managing the use of aggregation & classification Aggregation mechanisms: quantitative, qualitative & confidence

Royal NIVRA: Golden opportunity for the audit profession, Identify a way to contribute to systemic risk anticipation ComputationalAuditing.com Web platform for audit support: How to use that content? business wise by auditors, for auditors 8 Interactive audit documentation & business positioning: Building & uploading by fee-earning expert auditors Downloading & use by fee-paying engagement teams Broker-fee for the hosting platform provider Successfully positioned within Deloitte Trade in audit packs between member firms External auditors develop tailored packs & on-line services for clients internal audit department. Why? Marketing strategy of vendor lock-in Professional bodies of CPAs and standard setters upload high-level guidance packs la ISA & strict forms la Tax. Basis to be refined upon, but not overruled Audit Software: From Bench Warmer to Star Player, Royal NIVRA, de Accountant, March 2009, pp. 12-18, Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com ComputationalAuditing.com Web platform for audit support: How to use that content? society wise by auditors, for auditors 9 Interactive audit documentation & open pack-platform: Uploading by content providing expert auditors, using a dedicated content builder Downloading by engagement teams,

using a generic player to apply content Content is certified, published & hosted by A. an audit firms global and national office (layers) B. a professional body of auditors C. a standard setter or regulator each granting access rights to their members, ideally with content overlaying (A on top of B, B on top of C) Audit Software: From Bench Warmer to Star Player, Royal NIVRA, de Accountant, March 2009, pp. 12-18, Annotated translation into English by Dutch-American Translations & ComputationalAuditing.com Invitation to CaseWare & ACL: do you want to contribute to proposing a tailored version to AICPA & CICA? ComputationalAuditing.com 10 Recap Builder Player Platform-architecture Goal of the Builder Support in capturing audit methods Goal of the Player Goal of the Platform Support in applying audit methods Support in classifying audit methods Builder Player What keeps audit leaders up at night?, ACL, 2008 How to get the data? is not the challenge anymore. Today, audit analytics fully focuses on: How to use the data? & How to manage that use? Audit Automation as the Foundation of Continuous Auditing, Michael Alles, Alexander Kogan,

Miklos Vasarhelyi & Donald Warren, 16th WCAS, 2008 Aggregation & classification are key methods of using data, so lets have a look into how to manage aggregation & classification ComputationalAuditing.com Agenda 11 Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: What is the content? Web platform for audit support: How to use that content? Managing the use of aggregation & classification Aggregation mechanisms: quantitative, qualitative & confidence Royal NIVRA: Golden opportunity for the audit profession, Identify a way to contribute to systemic risk anticipation ComputationalAuditing.com Aggregation scheme for risk assertions (cf 20) 12 Managing the use of aggregation &

classification Risk summarization tables capturing assertion-based aggregation schemes What do the arrows mean? E.g. Table A1.2.1 accumulates risks regarding the assertion Systems that retain based upon underlying feeding questions such as E1.6 & classifies & propagates the accumulated risk to Table A1.2 & A1 to contribute to driving the configuring, via table S2, of audit tasks constituting the audit plan Yahoo! SiteBuilder + own plug-ins to specify, visualize & interact with aggregation links (W3C SVG) Experiments with Adobe Flex, MXML & Google Open Docs, considering CaseWares Open Engagement Website Building System ComputationalAuditing.com Expressible, in a similar way, in Deloittes Smart Audit Support, see: Computational Auditing, p.328 The arrow is an Audit Workflow operator Aggregation, Process Mining & Workflow 13

Managing the use of aggregation & classification Input: event log with journals, e.g. SAP A Output: smart flowchart B D C Analyzing 3232 cases, classifying casualties (red arrows): A. Invoice receipt without prior approval (2537x) B. Approval acquired after purchase completion (261x) C. Purchase order established for rejected request (9x) D. Handled order status skipping receipt (875x), etc. Design-time workflow vs. run-time workflow Based on: Towards a ComputerAssisted Audit Analysis of Business Processes: Process Mining as Tool for IT Auditors, Maria Bezverhaya, Emiel Caron & Piet Goeyenbier, de EDP-Auditor, NOREA, 2009 Pull signal from audit practitioners & IT audit educators Push signal from Technical University of Eindhoven, ProM, Fluxicon & Anne Rozinat ComputationalAuditing.com Computational Auditing: - focus on discovery of supercycle - framing stand alone workflows - connecting to 80 years of audit theory Ernst & Youngs Smart Flowchart Pilot Study

Top-level is Supercycle, or Top-cycle. Connects traditional cycles Fit recognized by Jagdish Gangolly, 2007-2008 25 M M DF 500 Lf Worlds strongest business process-oriented auditing theory: classical Dutch auditing theory (80+ years) & its best-fitting rigorous process theory: Petri nets tailored to the auditing domain 225 Cm Approach: Powerful and easy system to support practice, founded in theory Case by Hans Verkruijsse & EY team, 2005-2006 EYs evaluation report: - Clarifying. Refreshing. - Systematic framework Dft guides input preparation process (2009: new style) - Quantitatively motivated process decomposition 25 Fmd

Static: State S Balance Item 225 Dst Lft Cbft At 100 BF 25 D 500 L F C Ft LF LF 400 A 400 S 1,000 400 A 200 St BF Bft At

Bft 20 20 P Agent Legend A P A At Pt A 20 P Pt 20 P M: Majority Owner-Manager W S: Sales department B: Buy/Purchase department Dynamic: Transaction F: Financial department Wt W T T: IT department W: Warehouse manager Profit & Loss Item L: Labor/salary accounts P: Planning department Agents access is associated to: C: Creditor accounts 1. Transactions D: Debtor accounts 2. States Capital letter: authorized, legitimate access A: Application 3. Flows

Small letter: illegitimate access Case in Efrim Boritz CAATTs class, 2007-2008 14 ComputationalAuditing.com At Pt Managing the use of aggregation & classification New in 2009: Process mining; pilots by a Big 4, UvA.nl & CWI.nl Focus on topcycle discovery A Input: event log Wt Output: 1. As Is diagram (Ist) 2. Identify To Be (Soll) 3. Built-in audit analytics More on integrated audit analytics: Enterprise-level Process Documentation incorporating Automatic Audit Analytics, 2008 Deloitte/KU Symposium & follow-up with Raj Srivastava & EY CARAT Typology of Top-cycles 15 Managing the use of aggregation & classification Top-cycle: normative backbone of the business process-oriented audit approach

Typology/classification of top-cycles: ordered by the strength of the backbone Top-cycle concept & typology: Central result of integral evolution. Of business process-oriented Auditing Theory, Auditing Practice & Auditing Education. Over 60-80 years Unfortunately hardly translated into English previous slide: example supercycle Scientific foundation: rationally rigorous. With mathematical & computational formalization. Superbly suited for the digital age. Recognized as such in accelerating pace. Easy by new tech Limperg, Starreveld, Frielink, Blokdijk & Veenstra ComputationalAuditing.com 16 Industry classificationbased auditing concepts, norms & methods Managing the use of aggregation & classification Frielink et al. Supercyclebackboned Audit Approach Volumes 1, 2a, 2b, etc. Starreveld et al. Typology of Top-cycles

Decisive advantage of these concepts, norms & methods: no need to prove again in practice, since practice was part of the evolution process ComputationalAuditing.com Agenda 17 Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: What is the content? Web platform for audit support: How to use that content? Managing the use of aggregation & classification Aggregation mechanisms: quantitative, qualitative & confidence Royal NIVRA: Golden opportunity for the audit profession, Identify a way to contribute to systemic risk anticipation ComputationalAuditing.com Mechanism for quantitative aggregation ={ 18

At least one noncurrent inventory 2 Receivables + 3 Inventories 5 Assets or 5 Current Assets All three inventories are current XBRL US GAAP Taxonomy Type Polymorphism: Least Upper Bound in the Taxonomy Aggregation in XBRL: - Calculation linkbase - XBRL Formula Articulate XBRL Assurance functionality using a dedicated website builder (plug-ins) instead of handcrafting XBRL Formulas See: On Positioning XBRL Assurance Business Rules in a Computational Infrastructure for Modern Auditing, 2009, University of Kansas, Annual International Conference on XBRL Plug-in type polymorphism mechanism (transferable) from programming language into XBRL Assurance Builder & Player Domain-Specific Language (DSL) for auditing: Pacioli, developed by Dutch software partner in cooperation with national research center for mathematics and computer science in the Netherlands (CWI) & University of Amsterdam ComputationalAuditing.com Mechanism for qualitative aggregation: 19

How to aggregate weak spots in the Internal Control that are both irreplaceable and indispensable, e.g. weak spots in Segregation of Duties? Irreplaceable in the sense X-Raying Segregation of Duties: Support to Illuminate an Enterprises Immunity to Solo-Fraud with discussions & response, IJAIS, June 2008 Method locating Clarifies why & how weak spots in the SoD require who has too many a hot-line direct-top-level aggregation mechanism authorizations in one hand creating a For reasons of efficiency: establish a full dangerous opportunity for Top-of-iceberg aggregation as early as possible in the audit traceless embezzlement, solo-frauds: process (observation by William Kinney) jeopardizing the integrity 1. Madoff 2. Stanford of financial statements 3. Kerviel, etc. Solo-fraud free? Design, Implementation & Operation that there is no way for an external auditor to compensate its lacking or failing, while it is indispensable for a rationally justifiable approval Get it right at entry level Focal point in modern auditing? Launched at Accountant.nl by Jules Muis, Oct. 2009. Directly endorsed by Hans Blokdijk, Marc van Hilvoorde and others. Berry Wammes, CEO Royal NIVRA, directly stated the intent to position Get it right at entry level as the theme for the NIVRA spring 2010 debate series Continuous auditing web service (hosted via external auditor?) intercepts every Authorization Change Request to signal:

refuse human intervention required OK Efrims proposal (2008): Large-scale introductory study for this science-based method. As for new medicine. New method on top of Dutch auditing theory as incarnated in computational process theory. Collaboration with Canada. Identification of budget doubling when large audit firm steps in. Current status: pilots by Big 4 Dutch member firm ComputationalAuditing.com Mechanism for confidence-level aggregation (cf 12) 20 Based on: Sun, Srivastava & Mock, 2006 An Information Systems Security Risk Assessment Model, pp. 43-48 ComputationalAuditing.com This can be realized in Deloittes Smart Audit Support with a plug-in for Dempster-Shafer-Srivastava confidence-level computations Agenda 21 Automatic Aggregation in Auditing: with an Application to Systemic Risk Anticipation Web platform for audit support: What is the content?

Web platform for audit support: How to use that content? Managing the use of aggregation & classification Aggregation mechanisms: quantitative, qualitative & confidence Royal NIVRA: Golden opportunity for the audit profession, Identify a way to contribute to systemic risk anticipation Golden Opportunity Jan Helderman, President Royal NIVRA, Accountant.nl, Sept. 2009 The PCAOB and the Social Responsibility of the Independent Auditor Douglas Carmichael, Founding Chief Auditor of the PCAOB Early Warning System as Killer App for XBRL Assurance & Continuous Auditing: speeding up getting their Place & Future into Here & Now ComputationalAuditing.com Golden Opportunity Royal NIVRA: Preparing for an Audit Mandate to Contribute to Systemic Risk Anticipation, magazine, web & adopted in Sharing Knowledge-project 1. 2. 22 Proposed Solution An off-the-shelf system for tracking-and-tracing bar-coded products, configured for, and populated by XBRL tagged financial products A regulator-mandated auditor attests internal controls for the XBRL reporting channel to the new governmental systemic risk agency. Allowing for a continuous data streamfurther subjected to audit tests, sampling & monitoringwith on-the-fly automatic aggregation into systemic risk indicators (release 1.0: Bookstaber indicators)

Bailing out inflates moral hazard, early warning deflates Limpergs Theory of Rationalized Confidence More rigor on macro, more rigor on micro: use Dutch auditing 1. How far away? XBRL Assurance is closer than ever Instead of expecting more from XML, start expecting more from the builder-based approach to XBRL & continuous auditing 2. Release 1.0: matter of weeks or months, not years Jumpstart by cooperation of top-specialists Rick Bookstaber, Miklos Vasarhelyi, Raj Srivastava & Charlie Hoffman, and preferably in cooperation with a Big 4 audit firm Small step for XBRL & Continuous Dutch Auditing Day, hosted by Auditing, Royal NIVRA, November 25, 2009, quantum leap for the financial world agendas keynote & key discussion: ComputationalAuditing.com risk systems & systemic risk

Recently Viewed Presentations

  • Antibióticos en infecciones de vías respiratorias altas

    Antibióticos en infecciones de vías respiratorias altas

    Del Spinks A, Glasziou PP, Del Mar CB. Antibiotics for sore throat. Cochrane Database Syst Rev. 2013;11:CD000023. doi: 10.1002/14651858.CD000023.pub4 Beneficio clínico de antibioterapia en la faringoamigdalitis Del Spinks A, Glasziou PP, Del Mar CB. Antibiotics for sore throat.
  • CSCI 204 Introduction to Computer Science II

    CSCI 204 Introduction to Computer Science II

    assertion, by definition, is a statement that must be true at all times. An assertion in SQL is a boolean-valued SQL expression that must be true at all times. Defined by: CREATE ASSERTION . assertName. CHECK . cond; Condition may...
  • Food Chains and Food Webs

    Food Chains and Food Webs

    Types of Food Chains Aquatic- Water-related food chains with sea plants and animals Terrestrial- Land-related food chains with land plants and animals Predator & Prey Predator- An animal that captures and eats other animals Prey- The animal that is captured...
  • Safety and Health Programs Slide Presentation

    Safety and Health Programs Slide Presentation

    Provide training to all managers, supervisors and workers as well as contractors and temporary workers on: safety policies and procedures, program functions, emergencies, injury illness reporting, and their rights under the OSH Act. Ensure the training is provided in a...
  • The Alchemist

    The Alchemist

    FABLE Some well-known fables include: Aesop's fable, "The Tortoise and the Hare" Rudyard Kipling's The Jungle Book George Orwell's Animal Farm. ARCHETYPES An archetype is a pattern or model of an action, a character type, or an image that recurs...
  • Of Mice and Men

    Of Mice and Men

    Of Mice and Men A novella by John Steinbeck John Steinbeck (1902-1968) Born in Salinas, CA Father was a local official Mother was a schoolteacher she inspired his love of reading and writing John Steinbeck (continued) During summers, he worked...
  • Traverse Analysis - surveying.org.au

    Traverse Analysis - surveying.org.au

    Rod Deakin. THE KORUMBURRA SOLUTION. The Korumburra Cenotaph. 218 men from the district listed as having died during WW1 (165) and WW2 (53) Western Front casualties WW1. 1916 (39) 1917 (58) 1918 (24) Total (121) Gallipoli & Middle East casualties...
  • Marlin E. Rice Western corn rootworm (Diabrotica virgifera)

    Marlin E. Rice Western corn rootworm (Diabrotica virgifera)

    In each year of the model agricultural habitat patches are rotated, insect populations (N1) disperse, and reproduction occurs to form the next generation (N2) if insects land in corn. Simulations were initiated with 5 percent of corn fields infested with...