1/26/20 WIOA Performance Accountability: A Focus on Youth

1/26/20 WIOA Performance Accountability: A Focus on Youth Performance October 25, 2017 Where Are You? Enter your location in the chat window (lower left of screen) 2 Your Moderator: Sara Hastings

Unit Chief /Division of Youth Services DOL/ETA 3 Our Journey Together TA Series Two Day Kick-off Webinars Series Kick-Off: Where weve been, where we are now and where were going! Tuesday, October 24, 2017 at 11:30 AM ET WIOA Youth Eligibility Live Question and Answer Session Tuesday, October 24, 2017 at 2:30 PM ET Coming Together at the Table--The Power of Youth Committees to

Convene, Coordinate and Collectively Impact Youth Lives Wednesday, October 25, 2017 at 11:30 AM ET WIOA Youth Performance Accountability Wednesday, October 25, 2017 at 2:30 PM ET 4 Your Presenters: Evan Rosenberg US Department of Labor, Employment and Training Administration, Division of Youth Services Cesar Acevedo US Department of Labor, Employment and Training

Administration, Office of Policy Development and Research 5 Todays Objectives To provide state and local youth practitioners with what they need to know about performance accountability Definitions of participant and exit Primary indicators of performance for WIOA core programs and other workforce programs Statistical Adjustment Model and Negotiations Quiz Time Which of the following statements is true?

A. If an in-school youth becomes a participant on June 15, 2017, they are included in the credential indicator for Program Year 2016. B. If an in-school becomes a participant on June 15, 2017, they are included in the measurable skill gains indicator for Program Year 2016. C. A career readiness certificate counts as a positive outcome in the credential indicator. D. A youth in postsecondary education in the 2 nd quarter after exit, who also works a part-time job 6 hours a week during the second quarter after exit, is excluded from the median earnings indicator. 7 Guidance TEGL 10-16 Change 1, comprehensive, joint performance guidance published Dec 2016

TEGL 26-16, use of supplemental wage information guidance TEGL 3-17, WIOA Annual Performance Report Submission Data Validation guidance (Joint) to be published by end of 2017 Youth Participant Definition of a Participant: reportable individual who receives services other than self-service or information-only services and has satisfied all applicable programmatic requirements including eligibility determination For Youth this means: Eligibility determination Objective assessment

Individual service strategy Received 1 of 14 youth program elements Participation date isnt until the last of these items occurs Exit Program exit occurs when the participant no longer receives services for 90 days and has no additional services scheduled The date of program exit is determined after 90 days of no services have elapsed and no future services are planned. At that point the date of program exit is applied retroactively to the last date of service. The following services do not delay an exit: Follow-up services Self-service Information-only services

Common exit is encouraged but not required across DOL programs (no common exit with ED programs) Types of Youth Follow-Up Services Post-Exit There are five WIOA youth program elements that may be provided as follow-up services If any of these 5 services are provided after exit, they do not trigger new eligibility determination/ new participation and may be coded as follow-up services Supportive services Mentoring Financial literacy services Services that provide labor market information Postsecondary preparation activities

Primary Indicators of Performance Six Primary Indicators of Performance: Employment Rate 2nd Quarter after Exit -(Education/Employment for youth) Employment Rate 4th Quarter after Exit -(Education/Employment for youth) Median Earnings in the 2nd Quarter after Exit Credential Attainment Rate Measurable Skill Gains Effectiveness in Serving Employers Employment/Education/Training Rate 2nd Quarter after Exit Percentage of participants in education or training activities, or in

unsubsidized employment during the second quarter after exit. Successful outcomes include unsubsidized employment, secondary education, postsecondary education, and occupational skills training in 2nd quarter after exit Adult indicator only recognizes unsubsidized employment as outcome Unlike WIA, WIOA success in this indicator is only based on status in the 2nd quarter after exit regardless of status upon program entry. For example, if ISY is in high school at program entry and still in high school in the 2nd quarter after exit, they are a success in the indicator even though they were already in high school at program entry. Employment/Education/Training Rate 4th Quarter after Exit Percentage of participants who are in unsubsidized employment during the fourth quarter after exit from the program.

Successful outcomes include unsubsidized employment, secondary education, postsecondary education, and occupational skills training in 4th quarter after exit Adult indicator only recognizes unsubsidized employment as outcome Success based solely on status in 4th quarter regardless of status at participation or in 2nd quarter not a retention measure Median Earnings in the 2nd Quarter after Exit The median earnings of participants who are in unsubsidized employment during the second quarter after exit from the program. Only includes those in unsubsidized employment in

2nd quarter after exit Median is the number that is in the middle of the series of numbers, so that there is the same quantity of numbers above the median as there are below the median Credential Attainment Rate Percentage of participants enrolled in education or training program (excluding on the job training (OJT) and customized training) who attain a recognized postsecondary credential or secondary school diploma within 1 year after program exit. Only includes those who received training or education (excluding OJT or Customized Training) Credential can be obtained during the program or within 1 year (365 days) following exit

If participant obtains secondary school diploma or equivalent, they must also be employed or in an education/training program leading to a postsecondary credential within 1 year after exit Who is Included in Credential Attainment Indicator for WIOA Youth What is considered education or training varies based on specific WIOA program For WIOA Youth: All ISY are included Not all OSY are included; only those in the following are included: Occupational skills training program element Secondary education (at or above 9th grade level) Postsecondary education

Adult Education (at or above 9th grade level) YouthBuild Job Corps Credential Attainment Parameters Only included in numerator and denominator of credential indicator 1 time regardless of number of credentials attained Even if obtain secondary school diploma and another credential, only count 1 time in indicator Types of Credentials Secondary School diploma or recognized equivalent Associates degree Bachelors degree Occupational licensure

Occupational certificate, including Registered Apprenticeship and Career and Technical Education educational certificates Occupational certification Other recognized certificates of industry/occupational skills completion sufficient to qualify for entry-level or advancement in employment Postsecondary Credential Definition Highlights Awarded in recognition of an individuals attainment of measurable technical or industry/occupational skills necessary to obtain employment or advance within an industry/occupation Technical or industry/occupational skills based on standards developed or endorsed by employers or industry associations Neither certificates awarded by workforce development boards nor work readiness certificates are included because neither

document the measurable technical or industry/occupational skills Must recognize technical or industry/occupational skills for specific industry/occupation rather than general skills related to safety, hygiene, etc., even if general skills certificates are broadly required to qualify for entry-level employment or advancement in employment Secondary School Diploma Definition A secondary diploma (or alternate diploma) is one that is recognized by a State and that is included for accountability purposes under the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the Every Student Succeeds Act (ESSA). A secondary school equivalency certification signifies that a student has completed the requirements for a high

school education. The types of recognized equivalents, for those not covered under ESEA, are those recognized by a State. Further examples of secondary school diplomas/equivalents can be found in TEGL 10-16. Examples Credentials that Count Examples of Credentials that Meet the Definition: Certified Nursing Assistant (CNA) License Example of Occupational Licensure Automotive Service Excellence (ASE) Certification Example of Occupational Certification States must follow definition not feasible to rule on each individual credential

Examples Credentials that DO NOT count Examples of common certificates that do not meet the credential definition: Occupational Safety and Health Administration (OSHA) 10 Hour Course that provides awareness of job-related common safety and health hazards Work/Career Readiness Certificates Completion of an Assistive Technology training program (e.g., screen reading software) Measurable Skill Gains Indicator Percentage of program participants who, during a program year, are in an education or training program that leads to a recognized postsecondary credential or employment and

who are achieving measurable skills gains, defined as documented academic, technical, occupational, or other forms of progress, towards such a credential or employment. Only count in numerator and denominator 1 time in a reporting period (i.e., program year) regardless of how many skill gains are attained unless the individual has more than 1 period of participation in the program year The measurable skill gains indicator measures progress in a program year; it is not exit-based Measurable Skill Gains Note: Participants (who are in education or training) are included in the indicator regardless of how long they have participated in the program year Even if an individual is enrolled in June, s/he is still

included in the indicator, if the individual meets the parameters for inclusion in the indicator Programs should not delay enrollment or services to participants until a new program year even if programs believe there is insufficient time for the participant to make any type of measurable skill gain by the end of that program year. Who is Included in Measurable Skill Gains for WIOA Youth What is considered education or training varies based on specific WIOA program For WIOA Youth: All ISY are included Not all OSY are included; only those in the following are included: Occupational skills training program element

Secondary education (at or above 9th grade level) Postsecondary education Adult Education (at or above 9th grade level) YouthBuild Job Corps Note: same Youth participants included as in credential indicator this is different than in the Adult program Five Types of Measurable Skill Gains Achievement of at least one educational functioning level, if receiving instruction below postsecondary education level The three ways to measure EFL gains are in TEGL 10-16 Attainment of secondary school diploma or equivalent Secondary or postsecondary transcript for sufficient number

of credit hours Secondary: transcript or report card for 1 semester Postsecondary: at least 12 hours per semester or, for part-time students, a total of at least 12 hours over 2 completed semesters during a 12 month period Five Types of Measurable Skill Gains (continued) Satisfactory progress report toward an established milestone from an employer or training provider Passage of an exam required for an occupation or progress attaining technical/occupational skills as evidenced by trade-related benchmarks

Operational Parameters for Measurable Skill Gain for WIOA Youth Measured by the achievement of any of the 5 types of measurable skill gains No specific measurable skill gain types required for specific Youth participants Type of skill gain should be based on youths Individual Service Strategy Effectiveness in Serving Employers Effectiveness in Serving Employers System measure reported jointly across all core programs (i.e., youth program does not have a separate employer measure) (States choose 2)

Retention with the same employer in the 2nd and 4th quarters after exit Employer Penetration Rate Repeat Business Customer Rate Data Availability WIOA Primary Indicators of Performance PY 2016 PY 2017 PY 2018 PY 2019

PY2020 Oct. 2017 Oct. 2018 Oct. 2019 Oct. 2020 Oct. 2021 % Of Data Available on Report Delivery Date Employment Rate 2nd Quarter After Exit

0% 100% 100% 100% 100% Employment Rate 4th Quarter After Exit 0% 50%

100% 100% 100% Median Earnings 2nd Quarter After Exit 0% 100% 100% 100%

100% Credential Attainment Rate 0% 50% 100% 100% 100% 100%

100% 100% 100% 100% ESE - Retention With the Same Employer 0% 50% 100%

100% 100% ESE - Repeat Customer* 0% 100% 100% 100% 100%

ESE - Market Penetration 100% 100% 100% 100% 100% Measurable Skill Gains *ESE - Repeat Customer indicator will build to a 3 year cohort.

PY 2017 will be based on repeat customers in the last year. PY 2018 will be based on repeat customer in the last 2 years. PY 2019 will be the first year the indicator will be based on repeat customers in the last 3 years. 31 Overview of Negotiations Requirements WIOA 116(b)(3)(A) requires the establishment of levels of performance for each performance indicator: (iv) Negotiations shall take place in two year intervals, for the 1st two years (PY2016 and PY2017), and then 3rd and 4th years. (v) Consideration of four factors when negotiating levels of performance for each indicator;

(viii) The statistical model will be used to make adjustments to the negotiated State levels of performance for actual economic conditions and participant characteristics. 32 Negotiations Requirements The negotiated levels for each indicator are based upon: Comparison with adjusted performance levels of other States; Adjustments of levels using an objective statistical model; Continuous improvement of each State; and The requirements of the Government Performance and Results Act (GPRA). 33

Statistical Model Requirements The Model must consider States actual economic conditions, including; Differences in Unemployment Rate Differences in job losses or gains in particular industries The Model must also consider the characteristics of participants at the time of enrollment; Indicators of poor work history Lack of work experience Lack of educational or occupational skills attainment Dislocation from high-wage or high-benefit employment Low levels of literacy or

English proficiency Disability status Homelessness Ex-offender status Welfare dependency 34 Elements of the Model The model is required to include two types of elements: Participant Characteristics variables are presented as percentages of the total exiters for the particular cohort. Economic Conditions variables are presented as

percentages of total employment level by industrial sector and the State unemployment rate. 35 Elements of the Model (contd) In addition to required elements, other relevant characteristics of participants served are captured, including: Gender Age Race/Ethnicity Education Level

Employment Status at Participation For reference, details of the variables can be found here: https:// www.doleta.gov/performance/guidance/docs/WIOA_Statistical_ Model_Methodology_Report-6-24-2016.pdf https:// wdr.doleta.gov/directives/attach/TEGL/TEGL_26-15-AttachmentII_Acc.pdf 36 Implementing the Model: Baseline Indicators Those indicators where there are insufficient data yet to support the statistical model have been designated as Baseline Indicators.

Baseline Indicators, by core program title, include: For Title I: Measurable Skill Gains, Effectiveness in Serving Employers, Median Earnings (Youth program only) For Title III: Effectiveness in Serving Employers 37 Determining Success or Failure Is this my final answer? The model must be applied at the end of the program year to adjust the negotiated State levels of performance for: Actual participant characteristics, and

Actual economic conditions 38 Key Takeaways on Negotiation and the Statistical Adjustment Model for the Youth Indicators Keep in mind that many of the indicators are still in their baseline period Keep in mind that characteristics of hard to serve youth are factored into the statistical adjustment model When negotiating levels keep in mind: Median Earnings impact on youth who work parttime while in postsecondary or training Measurable Skill Gains impact for those who enroll

late in the program year Resources: Workforce Innovation and Opportunity Act, Pub. L. 113-1128 Workforce Innovation and Opportunity Act Statute https://www.gpo.gov/fdsys/pkg/PLAW-113publ128/pdf/PLAW-113publ128.pdf WIOA Joint Rule for United and Combined State Plans, Performance Accountability, and the One-Stop System Joint Provisions, 81 FED. REG. 55791 (Aug. 19, 2016) This Joint WIOA Joint Final Rule provides guidance for State and local workforce development systems https://www.gpo.gov/fdsys/pkg/FR-2018-08-19/pdf/2016-15977.pdf

Training and Employment Guidance Letter (TEGL) No. 10-16 Performance Accountability Guidance for WIOA Title I, Title II, Title III and Title IV Core Programs https://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=8226 40 Resources: WIOA Resource Page The Department of Labor (DOL), in coordination with the U.S. Departments of Education (ED) and Health and Human Services (HHS), has worked to prepare everyone for the implementation of WIOA. The WIOA resource page provides information and resources for States, local areas, non-profits and other grantees, and

other stakeholders to assist with implementation of the Act. www.doleta.gov/wioa Performance and Results Web Site The Performance and Results Web Site will assist you in understanding how performance is measured, reported, and evaluated at ETA. www.doleta.gov/performance 41 Post Polling Question If a youth is determined eligible on September 1, receives an assessment and ISS on September 12, and receives their first WIOA youth service on September 22, what is their date of participation? A. September 12

B. September 22 C. September 1 D. They have not yet met the criteria to be a WIOA youth participant. 42 Any Questions? Enter your questions in the chat window (lower left of screen) 43 Ways to Engage and Connect: DOL and your colleagues need to hear from you!

1st things 1st: Become a GPS member and use the Member Directory Share your work Submit a Resource on WorkforceGPS. Send us your lessons learned and successful strategies to share with the field to [email protected] Participate in Discussion Threads 44 45

Recently Viewed Presentations

  • Présentation PowerPoint - Assembly of First Nations

    Présentation PowerPoint - Assembly of First Nations

    ABSCAN isworkingwith Canada, with social finance leaders such as the J.W. McConnell FamilyFoundation, and withfinancialmarket experts to engage social and market sources of capital to meet the escalatingdemand for capital to renewhousingin Quebec, and, over time, withregionalpartners, across Canada. 3.1
  • Welcome to Maths

    Welcome to Maths

    Pitfalls to avoid. Teach a particular topic and then give them questions on that topic involving different contexts. Students know what topic they are doing so they just look at the question to see how to apply the skills they...
  • Forw ard- l ook i ng analysis, sustainability

    Forw ard- l ook i ng analysis, sustainability

    Review and consolidation of the building blocks. What is new in the revised conceptual framework . Structure of the conceptual framework. Revised conceptual framework model . Before. After . Building blocks consolidated to reduce duplication and overlaps - reduced from...
  • Innovativa produkter… räcker det?

    Innovativa produkter… räcker det?

    Targets customers by introducing products and services that are not as good as currently available products and services - but they are simpler, more convenient, good enough and …less expensive. Tid. ... Tror du att de stora spelarna har svårigheter...
  • Autumn Term Class Newsletter Teacher: Mrs Hunt LSO:

    Autumn Term Class Newsletter Teacher: Mrs Hunt LSO:

    Our topic will be Dinosaur Planet. We will be exploring where dinosaurs lived, looking at fact books and videos to find out information, and making our own fossils. In Maths this term we will focus on place value and representing...
  • Binary Instrumentation For Hackers - cve-search

    Binary Instrumentation For Hackers - cve-search

    Starting at first application IP Read a Trace from Application Code. Jit it, adding instrumentation code from inscount.dll. Encode the trace into the Code Cache. Execute Jitted code. Execution of Trace ends. Call into PINVM.DLL to . Jit. next trace....
  • Using Context Clues - 15faweb.blackbaudondemand.com

    Using Context Clues - 15faweb.blackbaudondemand.com

    Using Context Clues (to help you read with more understanding) When you come across a word you don't know, first try to sound it out. The detective looked for evidence to solve the mystery. ev-i-dence (try dividing the word up...
  • Staking a claim - WordPress.com

    Staking a claim - WordPress.com

    Develop a thesis statement that could focus an argument in response to each of the following prompts. Discuss why you think that the structure (open, closed, counterargument) you chose would be appropriate or effective. You need a thesis statement and...