2013 CTN Web Seminar Series QUALITY ASSURANCE AND RISK BASED/STRATEGIC MONITORING Presented by: Elizabeth Alonso, PhD, CCRA Maria Campanella, BSN, RN, CCRA Robert Lindblad, MD Thursday, September 19, 2013 Produced by: NIDA CTN CCC Training Office "This training has been funded in whole or in part with Federal funds from the National Institute on Drug Abuse, National Institutes of Health, Department of Health and Human Services, under Contract No.HHSN271201000024C." Objectives: Define risk based and strategic monitoring

Identify factors to consider when determining how and how often to monitor Identify types of monitoring models Identify a model that will work for you 2 NEW FDA GUIDANCE DOCUMENT A RISK-BASED APPROACH 3 New FDA Guidance Document Oversight of Clinical Investigations A RiskBased Approach to Monitoring Current guidance issued in August 2013 Overarching goal: to enhance human subject protection and the quality of clinical trial data by

focusing sponsor oversight on the most important aspects of study conduct and reporting. 4 Observations from CTTI Survey of Monitoring Practices Perception that FDA requires that on-site visits occur every 4-8 weeks Perception that appropriate monitoring means a 100 % review of 100% of data 5 New FDA Guidance Document Suggests that varied approaches to monitoring that include risk assessment

and strategic planning are more likely to ensure subject protection and overall study quality than routine on-site visits to all clinical sites and 100% data verification 6 New FDA Guidance Document Advises sponsors to incorporate alternative monitoring approaches and onsite monitoring to improve the quality and efficiency of trial oversight The FDA recognizes that both quality and efficiency can be improved 7 New FDA Guidance Document

Rationale: Improve efficiencies and effectiveness of monitoring by focusing on the most critical data elements Detect data anomalies in various modes of monitoring to ensure the quality and integrity of clinical trial data Maximize opportunities to monitor remotely using the available technology, electronic data capture (EDC) systems centralized monitoring by monitors, data managers and statisticians 8 New FDA Guidance Document Does not suggest a need for less oversight

or vigilance Advises focused efforts on identifying, preventing, or mitigating important and likely risks to data quality and processes Stresses findings should help determine if/when additional actions are needed to ensure human subject protection and data quality across sites 9 The path from Conventional to Risk-Based Monitoring SPONSOR MONITORING WITHIN THE NIDA CTN 10 NIDA Clinical Trials Network (CTN)

A platform for conducting multi-site clinical trials targeting substance abuse treatment Network Composed of Academic Regional Research Training Center(s) (RRTC), also referred to as nodes (currently 13) Approximately 240 clinical sites, all US sites Sites are affiliated with nodes for IRB oversight, site management, training support and QA 11 NIDA Clinical Trials Network(CTN) Clinical Coordinating Center (CCC): Quality assurance monitoring, safety monitoring, protocol document development, collection and maintenance of regulatory documents Data and Statistical Center (DSC):

Data collection in EDC system, protocol document development, data quality assurance, statistical analysis The EMMES Corporation, a CRO, has been contracted by the sponsor to conduct the activities of the CCC and DSC. 12 The scope of the CTN from 2007 Present 2007 2010 4 active trials (300-1200/trial) 36 sites requiring monitoring 2011 - present 2 active trials (as of Sep 2013) 4 trials in closeout: 3 trials in Q1-Q2; 1 trial in Q3 40+ sites requiring closeout and monitoring activities, some on multiple protocols

4 trials under development est. 11-15 additional sites (at least half nave to CTN research practices) 13 Clinical Trial A THE CONVENTIONAL APPROACH 14 7 Years Ago The Reality Clinical Trial A n = 1200 (original n of 400 changed to1200, during the trial) Primary endpoint changes in liver enzymes Nine sites, various geographic locations Substance Use Disorder population

Large number of assessments per participant Large amount of paper CRFs requiring SDV Ancillary pharmacogenetics study requiring additional consent form review 15 The Conventional Approach Visits to each site conducted quarterly 100% review of eligibility criteria 100% review of informed consent process 100% review of primary endpoint data 100% review of reported SAEs 100% review of all data, including progress notes for 10% of randomized participants 16

The Monitoring Burden Clinical Trial A Number of Visits Year Q1 2006 2010 Q2 Q3 Q4

Total 8 8 7 23 2007 10 9 11

9 39 2008 6 10 7 12 35

2009 10 11 8 15 44 2010 19 23

12 Total 45 61 46 54 43 195 2009

Qtr1 Qtr2 2008 Qtr3 Qtr4 2007 2006 0 10 20

30 40 50 60 Number of visits conducted 2010 28% of study visits in 17% of study time 17 Analysis of the Conventional Plan Relevance In 2010 - most of these visits were catch up visits reviewing old data Cost

Of these same visits in Q1-Q2 of the last year of the study, 28 of 42 (67%) visits were co-monitored (2 or more monitors sent to the same site at the same time-doubling cost of the visit) Value Discrepancies noted in data were not critical to either primary endpoint or safety 18 Analysis of the Monitoring Impact Change in number of subjects enrolled and the duration of the study resulted in a significant increase in cost associated with monitoring Inability to adequately monitor other simultaneously conducted trials due to a lack of resources Increased site staff burden with increased number of multiple day visits combined with continuous findings

Decreased staff morale, monitor burnout, and high turnover 19 Lessons Learned The Conventional Monitoring Plan did not work for this study Data errors noted were not timely nor were they critical to primary endpoint or safety Failure to identify problem trends early on, in particular those related to processes, resulted in repetitive site staff errors Catch up visits were expensive and not a good value 20

Clinical Trial B THE BEGINNINGS OF CHANGE 21 A Risk-Based Approach Evaluated and categorized retrospectively the monitored trials : the number of site monitoring visits conducted the amount of resources utilized the associated costs and impact of monitoring practices across all trials Process problems and data errors noted Developed an algorithm to determine a hierarchy of monitoring risk for each of these trials in order to

better develop a monitoring plan for future trials with similar characteristics 22 3 years ago The Challenge Create a Risk-Based Monitoring Plan that takes the following into consideration: Identify potential risks associated with trial Data collection factors (source document vs. direct data entry) Plan for adaptation of the plan during study based on specific site performance number of randomizations, number of protocol departures, incidence of adverse events, screen failure rate, frequent changes in staffing at sites Actively participate in National Team and Operational calls to address study issues across all sites in a timely manner 23

Created Algorithm to Assess Trial Risk and Monitoring Complexity Assess complexity of protocol design Trial Risk Study population Safety profile of investigational product Critical study procedures Novel outcome or adherence measures 24

Created Algorithm to Assess Trial Risk and Monitoring Complexity Assess complexity of protocol design Monitoring Complexity Participant assessment frequency and length Number and complexity of assessments Number of primary endpoints Number of participants enrolled Number of sites and research experience of each Expected rate of enrollment and length of participant

involvement Complexity of regulatory components (DEA, OHRP, IND) Volume of available source documents vs. direct data entry 25 Created Algorithm to Assess Trial Risk and Monitoring Complexity Sophistication of EDC system and Data Management Focus on processes and consistency Use errors as a window into process 26 Moderate Risks to Consider Multi-site trials, high need for consistency

Vulnerable population, high need for confidentiality Varying levels of Investigator and site staff experience in research Heavy assessment burden for primary outcome Pharmacologic therapy Occasional novel approaches to primary outcome measures 27 Low Risks to Consider Behavioral interventions (unless novel) Safety profile of pharmacologic agents well known Low to Moderate risk for related, unexpected serious adverse events Centralized data collection in EDC

28 A Different Kind of Trial Clinical Trial B n = 500 Primary endpoint measure of abstinence from all drugs and heavy drinking days Ten sites Large number of assessments conducted per participant Large amount of paper CRFs requiring SDV 29 3 Years Ago Monitoring Data Clinical Trial B

Number of Visits Year Q1 2010 Q2 Q3 Q4 Total 10

10 10 30 1 5 25 2011 9 10

2012 6 11 Total 15 31 17 11 15

72 Number of visits conducted 30 The Transitional Monitoring Plan Started monitoring trial using conventional plan Quarterly visits, etc. Relatively low risk trial: no pharmacotherapy, no invasive procedures/assessments, etc. On-site monitoring had identified no significant issues related to data quality Decision made to tailor Monitoring Plan to apply a new Risk-based approach

Monitoring resources were re-allocated based on risk 31 Comparison of Monitoring Visits Clinical Trial A 7 years ago 3 years ago Clinical Trial B 3 years ago 1 year ago 2010 2009 Qtr1 Qtr2

2008 Qtr3 Qtr4 2007 2006 0 10 20 30 40

50 Number of visits conducted 60 Number of visits conducted 32 Analysis of the new Monitoring Approach Limited ongoing monitoring efforts to: 100% review of informed consent processes, eligibility criteria, safety events and primary endpoint data Compulsory quarterly visits changed to visits

based on site specific triggers No change in error rate noted after frequency of visits was decreased during the last third of the trial Trial database locked on time Conclusion New strategy maintained trial integrity, safety, timely monitoring and morale of both monitors and clinical site staff 33 Clinical Trial C THE RISK-BASED PLAN 34 Last Year A Tailored Approach Clinical Trial C

n= 300 11 sites Primary endpoint number of cocaine use days as measured by self-report and corroborated by thriceweekly urine drug screens Started with a plan tailored to the identified clinical trial risk High risk trial: Substance Use Disorder population, novel treatment combination, Schedule II medications, moderate level of safety risk, increased direct data entry, low amount of SDV required 35 Last Year A Tailored Approach Monitoring Plan: 100% review of informed consent processes, eligibility criteria, a sample of randomly selected participants for 100% data review and a random selection of other

critical primary endpoint data Review reported SAEs and protocol departures remotely in real time Review a percentage of data within a site and across all sites, leveraging the EDC system to identify trends in reporting 36 Leveraging Experience and Technology Incorporated data quality features into a comprehensive monitoring plan Worked with the Data and Statistical Center to continue to leverage web based data capture

Real-time range validations Skip logic within the data system Data integrity checks Protocol violation review system built within the data system to immediately address corrective action plans 37 The Present Monitoring Data Clinical Trial C Number of Visits 2013 Year

Q1 Q2 Q3 Q4 Total Qtr1 2011 5 8

13 Qtr2 2012 Qtr3 2012 8 10 2013 22

9 30 19 Total 22 7 47 31 27

15 91 Qtr4 2011 0 10 20 30 40

50 60 Number of visits conducted 38 The Present Monitoring Data Clinical Trial C Modes of Monitoring/Phase Year InitiationOn Site InterimOn Site InterimRemote

(TLFB) CloseoutRemote Initiation-On Site Interim-On Site Interim-Remote (TLFB) Closeout-Remote Total 2013 2011

11 2 2012 1 32 14 9 11 11

31 43 25 11 91 2013 Total 12 13

47 2012 2011 0 10 20 30 40 50

60 Number of visits conducted 39 Clinical Trial C - Conclusions Appropriate to tailor the sampling rate and type of visits conducted (on-site vs. remote) according to the phase of the trial (actively enrolling vs. active and follow up phases) Rate of site visits appears appropriate for the clinical trial Based on identified issues, study conduct, error rates, etc. Continued to be assessed throughout the trial

40 Clinical Trial C - Conclusions Cont. Analysis of Proactive Approach Recommending additional data quality checks based upon real-time monitoring findings Reviewing and assessing reported PVs remotely in real time provided window into process Allowed opportunity to address issues quickly, provide retraining and reduce the rates of recurring violations 41 An Adaptive Risk-Based Monitoring Plan LOOKING AHEAD

42 Targeted and Random Data Sampling Use the same criteria applied to Clinical Trial C Determine critical variables associated with participant safety, primary outcome and overall trial integrity Identify critical elements of the trial that are at risk for error in interpretation or completion (e.g., informed consent, study assessments, medication dispensing) 43 Enhancing The Plan On-site visit soon after first participant is

randomized More frequent visits in earlier stage of implementation, then reassess on a siteby-site basis Adapt overall plan with every protocol amendment, as necessary Adapt plan per site, as necessary 44 Risk Based Monitoring Benefits Adaptive and Tailored Monitoring Plans are allowing us to better utilize monitoring resources Review of ongoing trials have not identified problems with data quality or monitoring frequency Proactive approaches are identifying trends in data earlier and allowing for early corrective action

Focus on timely visits and site process improvement 45 Some Potential Risks Adjustments are to be expected Check and re-check assumptions about a trial and associated risk levels for suitability Need for prompt analysis of trends, re-evaluation, and adjustment to the plan Protocol specific training requirements Fewer visits requires properly trained study teams Lapse in sufficient and timely training can be disastrous High reliance on effective site management Sufficient oversight is critical Consider contingency management plans for staff turnover

Document procedures in SOPs Maintain up-to-date Manual of Procedures 46 MONITORING MODELS 47 ICH E6 and ISO Guidelines Both ICH E6 and ISO advise sponsors to assess and consider the objective, design, complexity, size, and endpoints in order to determine the extent and nature of monitoring for a given trial 48

Choosing the right model Number of sites Number of participants Length of trial Experience of research staff Complexity and number of interventions Complexity and number of assessments Vulnerability of study population

49 Centralized vs. On-Site Recent retrospective review of on-site monitoring findings from a multi-center international trial suggests that centralized monitoring activities could have identified more than 90% of the findings identified during on-site monitoring 50 Centralized vs. On-Site Centralized Efficient Cost effective Allows for Targeted and

Random selection of data Automated data integrity queries Ability to see and compare trends both within sites and across sites On-site A window into processes Verification of PI oversight, level of staffing and site logistics Reassurance that someone is personally watching the site Staff compliance with site

SOPs 51 Centralized AND On-site Take advantage of the option to do both Identify those things that have to be done on-site (e.g., ICF reviews, IP accountability) Identify other things that can be done remotely (e.g., lengthy and repetitive assessments, internal audit trail checks) 52 LEVERAGING THE ROLE OF THE MONITOR WITHIN THE CTN

53 The CTN Model Comprised of four distinct entities that monitor trial integrity and site performance CCC (Sponsor Monitors) Node (Local Monitors) DSC Site 54 Managing Risk Through Monitoring Sponsor (CCC) Monitors On-site visits to all sites Overview of protocol implementation

practices across sites Experience with CTN trials Can implement cross protocol risk-based and adaptive monitoring and centralized reviews Local (Node) Monitors Extensive experience with CTN trials Knowledgeable about local site practices (e.g.,SOPs, state laws, IRB) Established relationship with and proximity to sites

55 Site management and a view into process COMPLETING THE PICTURE 56 The Local Monitor/Protocol Manager Works on behalf of Node Study PI & Site PI Works closely with the study site unique perspective Often serves as liaison between Node/University and study site team More frequent visits/contacts than CCC/EMMES Some visits may be remote

May be responsible for regulatory submissions (correspondence with Local IRB) Familiarity with local personnel and procedures 57 The Local Monitor/ Protocol Manager Can help to fill in the gaps with close monitoring/support focused on local processes and procedures 58 Local Monitor/ Protocol Manager: Areas of site support Training & re-training of staff Addressing staff turnover

Training site team in protocol & MOP changes Documentation of staff responsibilities, qualifications & training Local SOPs SOP development & revisions Streamlining procedures for greater efficiency Allocation of site personnel/resources 59 Local Monitor/ Protocol Manager: Areas of site support Protocol deviations Ensuring proper documentation and reporting Corrective and preventive action plans (CAPAs) Identification of trends

Safety events Identification, tracking, documentation, and follow-up 60 Local Monitor/ Protocol Manager: Areas of site support Regulatory compliance Prompt review of Reg. Binder updates Ensuring protocol departures & safety events are reported to Local IRB, as appropriate Support in preparing IRB submissions: initial, amendments, and continuation reports Data review

Review data reports to ensure prompt resolution of queries or missing data Greater focus on forms/processes not reviewed by CCC/EMMES Review of Checklist/Progress Notes & Contact Logs Intervention documentation Service utilization data 61 Where it all begins INTERNAL QUALITY ASSURANCE 62 Internal Quality Assurance Quality Assurance begins with internal

monitoring and the expectation of a high standard of data quality and regulatory and GCP compliance. A culture of Quality is the foundation of a successful site. 63 Internal Quality Assurance: Strategies Informed Consent Documentation Real-time review of completed Informed Consent documents for completeness and accuracy Allows for prompt corrections by staff and/or participant Q: Are all printed names, signatures, dates,

and in/out responses present and correct? Can be cross-checked by another staff member (if available) 64 Internal Quality Assurance: Strategies Accuracy of data entry/abstractions Cross-check or double-check Regular review of data reports to ensure prompt resolution of data issues Missing Forms, Missing Values, Integrity Data Queries & others 65 Internal Quality Assurance: Strategies

Proactive approach to minimizing errors: Identification of likely or common errors Implementation of targeted strategies to increase accuracy Examples: Use of stylus pen in conjunction with a tablet Distinguishing ICF versions with different colored paper Use of checklists for complex or multi-step processes Reference or guidance documents displayed in key locations (e.g., steps for processing a urine test displayed in lab area) 66

FACTORS TO TRIGGER MONITORING ACTIONS 67 Factors Determining Monitoring Complexity/length of a trial Implementation (soon after first participant randomized) Volume of source to data verification Sites nave to research or the CTN Performance indicators Error rates from remote monitoring Due Diligence 68 Q&A Session

Alternatively, questions can be directed to the presenter by sending an email to [email protected] 69 Survey Reminder The NIDA CCC encourages all to complete the survey issued to participants directly following this webinar session, as this is the primary collective tool for rating your experience with this and other webinars, and for communicating the interests and needs of CTN members and associates. Upcoming Webinar ADAPTIVE RESEARCH DESIGN FOR SUBSTANCE ABUSE CLINICAL TRIALS Wednesday, October 23, 2013 1:00 pm to 2:00 pm ET

70 A copy of this presentation will be available electronically after this session. http://ctndisseminationlibrary.org 71 Thank you for participating. NIDA CTN Web Seminar Series 72

Recently Viewed Presentations

  • Fundamentals of Electric Circuits

    Fundamentals of Electric Circuits

    A current that varies sinusoidally with time is called Alternating Current (AC) Mains power is an example of AC. Direction of current. ... Fundamentals of Electric Circuits ...


    ASYMPTOTIC NOTATION. 1.1. What is the meaning of 'asymptotic'. Consider a curve such as y = 1 / xAs x increases, y decreases. The positive x-axis is an asymptote of the curve. As you continually increase x towards infinity, the...
  • Catalysts for Change Principles and standards for school

    Catalysts for Change Principles and standards for school

    3. If at least one angle of a parallelogram is a right angle, then the parallelogram is a rectangle. 4. If a quadrilateral is a parallelogram, then its opposite angles have the same measure and its adjacent angles are supplementary....
  • Resisting Peer Pressure - Boone County Schools

    Resisting Peer Pressure - Boone County Schools

    Group Pressure. Think of the last time you felt pressured by a group to do something that you did not want to do. With your situation in mind, turn to page 64 in your LIfeSkills book and complete worksheet 19.
  • Performance engine preparation Cam timing Check accuracy of

    Performance engine preparation Cam timing Check accuracy of

    Difference is TDC Performance engine preparation Timing a cam using the intake centerline Install degree wheel on damper Dial indicator on #1 lifter (retainer for OHC) Zero indicator at valve wide open Back engine up ¼ turn, then clockwise to...
  • 'We will walk on our own feet; we will work with our own ...

    'We will walk on our own feet; we will work with our own ...

    The roots of the American philosophy ran deep into German and English Romanticism. From German philosophers such as Fichte and Herder, it received its mystic impulse; from Goethe, Novalis, Jean-Paul, Heine, and the other great German Romantic poets it acquired...
  • The US responds to War - mrsziemnik.net

    The US responds to War - mrsziemnik.net

    The US responds to War How did the US try to remain neutral, but get pulled into war? Isolationism vs. Internationalism The Nye Committee finds that munitions makers and bankers actually profited off of WWI—some felt US went to war...
  • CCSS Pilot Report card Successful Practices

    CCSS Pilot Report card Successful Practices

    Time to process and collaborate. With a colleague or grade level team, dialogue about the following: What structures are already in place in your classroom to assist you in observing and collecting data to measure growth with the Successful Practices?