Investing in Innovation (i3) Fund Summary -

The Centre for Literacy Fall Institute 2012
Social Finance and Innovation
for Adult Basic Learning:
Opportunities and Challenges
October 14-16, 2012
Saint John, NB
Government Interest in Models of
Social Innovation and Alternative
Funding Models
October 15, 2012
Johan E. Uvin
Deputy Assistant Secretary for Policy and Strategic Initiatives
Office of Vocational and Adult Education
U.S. Department of Education
What is the context?
Adult training, skills, and employability
Architecture


Coordination and Efficiency Challenges



2
Nine (9) federal agencies spent approximately $18
billion to administer 47 employment and training
programs (including adult education) in 2009 The
Departments of Labor, Education, and Health and
Human Services (HHS) largely administer these
programs. - GAO 2011.
Inefficiencies and data challenges due to multiple
administrative structures.
Overlap is a reality: similar services to similar
populations.
Context (2)
Adult training, skills, and employability
 Evidence base is growing but still modest
 Only five (5) programs have had a net impact study
completed since 2004.
 No definite word on best in class but multiple
promising approaches are emerging.
3
 Performance and quality are uneven
 Performance of status quo/legacy programs and
systems (particularly those focused on vulnerable
populations), on average, is not optimal.
 There is great variation between and within States
and programs in outcomes and costs.
.
Context (3)
Adult training, skills, and employability
 Limited access due system capacity constraints
 Demand for services (e.g., ESL, training, vocational
rehabilitation) is often greater than supply..
 Systems are not well-equipped to achieve favorable
outcomes at scale through traditional service delivery
and funding.
4
 Growth in performance-based funding models
 After federal technical assistance effort, 26 States
implemented performance-based funding voluntarily in
adult education and 2 more are getting ready to do so.
 Similar developments in postsecondary education in
select States.
Context (4)
Adult training, skills, and employability
 Fiscal outlook is uncertain
 New investments … Temporary boost in funding
through the American Recovery and Reinvestment Act
of 2009. New resources ($2 billion over 4 years) for
TAACCCTI to build community college capacity.
Increase in Pell grants.
 New challenges … Decline in select State
investments in adult education, higher education, and
K-12. Deficit reduction needs and sequestration.
5
Context (5)
Adult training, skills, and employability
 Policy interest/support for reform/ innovation
 Federal policy support aimed at development of products
and services that lead to significant improvements in
outcomes for individuals beyond the status quo through
the use of flexibility and competitive funding to stimulate
innovation tied to evidence tiers
6
How We Talk About Innovation
Innovation
Greater Impact
Invention
product, process,
strategy, or approach
that improves
significantly upon the
status quo and
reaches scale
Baseline
Trend
Scale
7
Note: The definition of innovation on this slide is presented as an overview of the concept, not as a specific definition in
any of the Department’s innovation programs.
Context (6)

Federal policy efforts in support of innovation
in States and local organizations






Federal policy issues that may hinder
innovation in States and local organizations


Eligibility, allowable fund use, and other
Recent efforts


8
National activities (e.g., P2P)
State leadership activities (e.g., PD for IET)
Non-formula, competitive funding (e.g., budget requests)
Flexibility requests/waivers (e.g., disconnected youth)
Other (e.g., prizes)
Pre-K-12: RTT (State, ELC, Districts) & i3
Workforce Innovation Fund
Why is the government interested?
 What is attracting the U.S. government to
models of social innovation and to alternative
funding models (within and outside the
designation of “social finance”) to address
various social problems, in particular those
related to adult training, skills and
employability?
9
What is the Administration interested in?
Pay for Success Model
 Federal programs must be measurably effective
and designed to do more with fewer resources.
 Pay for Success is an innovative way of
partnering with philanthropic and private sector
investors to create incentives for service
providers to deliver better outcomes at lower
cost—producing the highest return on taxpayer
investments.
 The concept is simple: pay providers after they
have demonstrated success.
10
What is the Administration interested in?
Pay for Success Model
 Pay for Success bonds:
 Engage philanthropic and private sector investors to
deliver better outcomes.
 Can help achieve better outcomes in many program
areas.
 Support better outcomes for federal, state, and local
governments.
 Minimize risk to the government.
 Special funding provisions allow existing
programs to support pay for success bonds.
 Certain programs are likely candidates for pay
for success bond financing.
11
Why is the government interested?
 Get at best possible outcomes at greater scale
 Individual – prosperity; Business and Industry –
competitiveness; Government – reduced risk, greater
returns, savings; Society – reduced social ills
 Create opportunity to provide more support to
most vulnerable in face of fiscal cutbacks
 Pay for Success offers a new approach to invest in
services for vulnerable populations while saving.
 Offer a new, alternative vehicle for targeting
12
limited resources to achieve a positive,
measurable outcome.
 Bring into play and engage non-traditional
stakeholders/investors.
Why is the government interested?
 See if potential benefits can be realized …
 Overcoming the challenges of blending programand population-specific government funding
 Creating incentives for social innovation, improved
workforce outcomes, public sector cost savings, and
efficiency gains
 Identifying effective preventative services
 Allowing for more rapid learning about which
strategies are most effective
 Transferring social investment risk from the
government to private investors
 … through carefully designed pilots
13
Why is the government interested?
Selected Pay for Success Pilots
City, State, and National Pilots Under Way
 Massachusetts
 Maryland
 New York City
 Department of Justice Pilot
 Department of Labor Competition
National Pilots Being Planned
 Department of Education Career and Technical
Education Budget Proposal
Exploring Potential and Issues
 Interagency effort re: disconnected youth
 Adult education
Why is the government interested?
Department of Labor Pilot
 $20 million Department of Labor Competition
(In Progress)
 Objectives
 Test a model for government investment in
preventative and innovative service delivery models
that transfers risk to the private sector.
 Learn whether the Pay for Success concept is feasible
in the workforce development policy arena.
 Determine whether preventative social services
complementing workforce development programming
“pay off.”
15
Why is the government interested?
Department of Labor Pilot
Critical Elements of the Pay for Success Model








A well-defined problem and target population
A flexible and preventative service delivery strategy
Committed funds from investors to cover operating costs
Well-defined, achievable outcome targets
Outcome measurement methodology
Project timeline for outcome achievement dates
Financial model showing public sector savings
Payment arrangement details between the applicant and
intermediary
Why is the government interested?
Department of Labor Pilot
Required Partners
 State/local/tribal government agency
 The intermediary
 The investors
 The independent outcome validator
Other entities involved
 Service Providers
 U.S. Department of Labor
Why is the government interested?
Department of Labor Pilot
Award Amount:
 Up to $20 million in grant funds, with awards ranging up to
$12 million for approximately 1 - 3 grantees
Period of Performance:
 Awards expected in Spring 2013
 Total technical grant performance period - 48 months
 36 months for start-up, implementation, and delivery of
intervention
 6 months for outcome measurement and documentation
 6 months for DOL approval and payment
Why is the government interested?
Department of Labor Pilot
Eligible applicants
 State Workforce Agencies eligible for assistance under Title I
of the Workforce Investment Act (WIA) of 1998
 Local government agencies responsible for workforce
programs under Title I of WIA
 Federally recognized Indian tribal governments
Why is the government interested?
Other Efforts
Planning Under Way for a PFS Pilot in Career and
Technical Education
 Budget (2013) and reauthorization proposals include
approximately $100 million for local innovation and reform
including a Pay for Success pilot.
 Given the importance of alignment with and responsiveness
to labor market needs and the skill demands that business
and industry have, there is great opportunity to engage
business and industry as investors.
 There are sufficient intermediaries for a pilot.
 There is sufficient capacity in the market place for
independent outcome validation.
Why is the government interested?
Department of Justice
 Three Pay-for-Success Grant Opportunities
under the Second Chance Act
 Helps individuals exiting jail or prison make a safe and
successful transition to the community.
 Eligibility for these grants is limited to states, local
governments, and Indian tribes.
 The three grant programs are:
 Adult Offender Reentry Program for Planning and
Demonstration Projects; Reentry Program for Adult
Offenders with Co-Occurring Substance Abuse and
Mental Health Disorders; Family-Based Offender
Substance Abuse Treatment and Reentry Program.
21
Why is the government interested?
Key questions in exploring possible Adult
Education PFS Pilots
 Some key questions from other sectors that are highly
relevant (Adapted from McKinsey)
 Are the current remedial costs on a scale that would make the
economic savings generated by substituting a preventative solution
meaningful? And, what would be the features of such preventative
solution?
 Are there proven preventative interventions to help the target
population?
 Are there enough service providers who could scale these
interventions?
 Is there a meaningful number of constituents who could be served?
Why is the government interested?
Key questions in exploring possible Adult
Education PFS Pilots
 Is the evidence base sufficiently strong?
 Relationship between literacy and numeracy and multiple
economic and social outcomes is observed in the
population but evidence re: net impacts (causal effect) of
participation in programs is thin/emerging for selected new
models (IET programs such as I-BEST).
 Changes in net fiscal contributions for adults with and
without high school credentials are well-documented,
conclusive and strong (from – to +).
 Can the financing model work in adult education?
 How do we best address selectivity threats to ensure
the most vulnerable can access services?
Why is the government interested?
Key questions in exploring possible Adult
Education PFS Pilots (2)
 Is there sufficient intermediary capacity?
 There is limited intermediary capacity in adult education overall.
 There would be a need to incentivize development of the specific
intermediary capacity required or to motivate intermediaries in related
sectors to expand into adult education.
 How do we make sure federal, State, and local staff
expertise is developed quickly?
 How do we motivate private investors?
 What type of technical assistance is needed?
Why is the government interested?
Key questions in exploring possible Adult
Education PFS Pilots (3)
 What are the specific costs and possible benefits
(McKinsey)?
 Benefits to taxpayers
 Costs of providing the services
 Costs of the PFS structure
 Net savings
 Savings to taxpayers
 Returns to investors
 Success fees to service providers and intermediary/-ies
Why is the government interested?
Next steps re: adult education PFS Pilot
 Think of this in phases with “proof points” being the focus
of the initial pilot stage (McKinsey)
 This year, as part of the pilot phase, we plan to:
 Establish a cross-functional design team
 Conduct further feasibility research and deeper research
into the economics
 Identify possible investors, intermediaries, and independent
outcome validators
 Develop design proposal and technical specifications
 Draft a solicitation document
Appendix
Additional Slides
27
Role of Evidence in Innovation
Tiered System of Evidence Standards (i3)
 Types of Innovation Grants (i3)
1. Development
2. Validation
3. Scaling
28
OVAE and Innovation
Purpose
Focus
Funding
29
Applicants
For several years now, OVAE has proposed and the
President’s budget has included requests for AEFLA and
Perkins to provide competitive grants to applicants with a
record of improving student achievement, attainment or
retention in order to expand the implementation of, and
investment in, innovative practices that are demonstrated
to have an impact on student achievement and outcomes
and on enrollment in postsecondary education and
training and on completion and placement rates.
OVAE’s focus has been on career pathways system
development including bridge programs, concurrent
enrollment programs, onramp programs, and enhanced
ASE programs including face-to-face technology-enabled
learning opportunities. Growing interest in Pay for Success.
Non-formula funding for competitive grants to be awarded
through a national competition. Just over $100 million in
Perkins in FY2013 and $15 million in AEFLA (WIF). In
addition, $200 million to begin scaling Career Academies.
Varies. States and local eligible providers or consortium.
Development Grants:
Require Evidence to Support Intervention
 Theoretical support for the proposed practice, strategy, or
program that is based on research findings or reasonable
hypotheses, including related research or theories in
education and other sectors
and
 Some empirical evidence of the promise of the proposed
practice, strategy or program based on prior
implementation and evaluation of something similar, albeit
potentially on a limited scale or in a limited setting
Note: We expect most grant proposals in AEFLA and PERKINS to be
development-type proposals
30
Validation Grants:
Require “Moderate Evidence” of Effectiveness
 Validity of the evidence
 High internal validity and moderate external validity
or
 High external validity and moderate internal validity
 Minimum size of evidence base
 At least one well-designed experimental or quasi-experimental study
May have small sample sizes OR may fail to demonstrate equivalence
between the intervention & comparison groups, but has no other major
flaws
or
 A correlational study with strong statistical controls for selection bias
and for discerning the influence of other potential confounds
31
Note: We expect some validation-type grant proposals in AEFLA and PERKINS
Scale-up Grants:
Require “Strong Evidence” of Effectiveness
 High internal validity of the evidence
 Studies designed and implemented in ways that support causal
inference
 High external validity of the evidence
 Studies based on a sufficient representation of participants and
settings that the findings support scaling
 Minimum size of evidence base
 More than one well-designed and well-implemented experimental
or quasi-experimental study
OR
 One large, well-designed and well-implemented multi-site
randomized controlled trial
Note: We don’t expect many scaling grant proposals in AEFLA and PERKINS 32
Caution:
Not All Associations Support Causal Inferences
(Mis-) Interpretive Statement:
Based on the evidence, early
care & education programs
should reduce pupil teacher
ratios in order to lower the
expulsion rate.
Problem: There are competing
explanations for why
expulsions increase with
increases in pupil-teacher
ratios
33
WWC Standards Would Apply to Causal
Designs
Eligible Designs
• Randomized controlled
trials (RCTs)
• Quasi-experimental
designs (QEDs)
Potentially Eligible Designs
• Regression discontinuity
(RDD)
• Single case (SCD)
Ineligible Designs
• Anecdotes and
testimonials
• Case studies
• Descriptive
• Correlational
34
Draft Stylized Depiction of Formula &
Innovation Funding Relationships
Formula Funding
National Activities
Funding
National Competitive Funding for
Local Innovation and State Reform
Federal
State
State Reform
Initiatives
Local
Participation
of Local
Eligible
Providers
Effective Practices
35
Development
Scaling
Validation