CCB Discussion| Avionic Systems Div.

Risk Compensation With an Economic Model of Optimization of Innovation and Risk CCB Discussion| Avionic Systems Div. Robert Shuler |June 17, 2013 Goals of Presentation Introduce engineers to risk compensation Generally engineers are not expected to read economics & psychology journals Relation to other theories and existing practices Striking and large examples of unexpected outcomes Present a quantitative equation for optimization More appealing and useful to engineers than vague psych/econ theories Quantitative rationale for reliability decisions, including some post-Columbia actions Summary of derivation, details are in paper http://mc1soft.com/papers Brief tutorial on applying the equation Discuss a few of the examples in terms of equation parameters Exercise: decision / alternatives on a hypothetical space project commercial passenger carrier for an orbital tourism / hotel facility 1969 quant safety goals not adopted bec of uncertainty in risk calculations 1986 estimates of catastrophic failure so high would threaten political viability of programs Background Theories Risk Analysis (Asipu 3200 BC, max release scenario for Nuc. Pwr. 1950s, probabilistic assessment post 1967 pre-2008 pre-2008 ratings ratings of of MortgageMortgageBacked Backed Securities Securities & & Credit Credit Default Default Engineering / Statistical hypothesis combination of estimated probabilities Swaps

Swaps Apollo fire) http://www.rff.org/RFF/Documents/RFF-Resources-172_Risk_Assessment.pdf http://www.public.iastate.edu/~jdwolt/570X/2_RA%20History%20&%20Perspective.pdf http://www.econ-it2.eu/en/training/4-risk-management/4-1-the-essence-of-risk/4-1-1-the-origin-of-risk-analysis/ assumes a fixed mission profile (i.e. user behavior does not change typically to an engineer, human factors means clarity and usability of controls, etc.) Risk Management barred by from NASA barred by law lawMarkowitz from using using most (de Mere, Pascal, Fermat, Bernoulli, de Moivre,NASA Bayes from 17 th century, 1952)most common common strategies strategies insurance, insurance, futures, futures, Management hypothesis identification, assessment & prioritization of risks derivatives derivatives http://www.slideshare.net/ricardo.vargas/ricardo-vargas-historico-gerenciamento-riscos-ppt-en http://www.moaf.org/publications-collections/financial-history-magazine/93/_res/id=File1/Risk%20Management.pdf transfer, avoid, reduce or accept assumes user behaves only as legally constrained Includes insurance, futures & derivatives, e.g. pre-sale of farm crops Seatbelts Seatbelts do do save save lives, lives, (Peltzman 1975) but but not

not as as many many as as expected Economic hypothesis humans & organizations optimize economic value expected Risk Compensation risk may partly adjust when an improvement is made Risk Homeostasis (Wilde 1982) e.g. e.g. More More aggressive aggressive driving driving Psychological hypothesis humans adjust behavior to maintainwith risk anti-lock with anti-lock brakes brakes also also condoms, condoms, bike bike helmets, helmets, skydiving skydiving improvement is wasteful gear gear Unexpected Outcomes TRAFFIC SAFETY Montana No-Speed-Limit 1995-1999 After 4 years, Montana recorded its historical low of number of accidents on affected roadways http://www.hwysafety.com/hwy_montana.htm Since then accident rates have begun to rise again German Autobahn accident rates are lower than USA rates No-Fault Auto Insurance Leads to 6% increase in traffic fatalities

Cohen & Dehjia, J. of Law & Economics 2004 Seat Belts vs. Air Bags Levitt & Porter, Rev. of Econ. & Stat. 2001 Cost per life saved: Seat Belts $30k Air Bags $1.8M http://www.rand.org/pubs/research_briefs/RB9505/ index1.html 1 1 Unexpected Outcomes LARGE GOVERNMENT PROGRAMS War on Drugs Just Say No DARE Pioneered in 1970s by Richard Evans at University of Houston Nancy Reagan phrase 1982 Testing by marketing experts shows: Just Say No increased teen interest in drugs after exposure, promotes idea that other teens are using drugs usage has increased! DARE messages from police / authority not as effective as peer messages, e.g. publication of actual drinking statistics near dorms/frats on college campus reduced drinking Costs: Over $1 trillion & hundreds of thousands of lives $33 billion for Just Say No, $20 billion to fight in home countries, Columbia violence moved to Mexico, now directly affects

border regions and tourism, $121 billion to arrest and $450 billion to incarcerate non-violent offenders wiki/ List_of_countries_by_incarceration_rate per 100,000 1. USA 716 5. Rwanda 527 8. Russian Federation 502 92. United Kingdom 151 123. China 120 Unexpected Outcomes LARGE GOVERNMENT PROGRAMS #2 War on Cancer Begun by Nixon in 1971 Inspired by successful Moon program Promised cure by 1976 Some Some economic economic factors factors may may be be involved involved but but clearly clearly aa technically technically difficult difficult problem problem

NYT Advances 4/23/2009 Cancer Change in death rate or incidence: -5% death rate (since 1970s) Heart disease -64% death rate (since 1970s) Flu & Pneumonia -58% death rate (since 1970s) Smoking -54% incidence (since 1960s) http://www.infoplease.com/ipa/A0762370.html Illegal drug use http://www.umsl.edu/~keelr/180/trends.html +6% incidence (since 2002) Unexpected Outcomes MISCELLANEOUS ACTIVITIES Commercial Air Transportation People will pay more for safety in the air Carlsson 2002 Consumers learn about unobservable safety from flight outcomes Hartmann 2001 Accidents adversely affect demand for other carriers Airlines profits are greater if they are able to choose their optimal maintenance provision Airlines AND consumers prefer an independent safety certification rather than an FAA minimum makes it more profitable to provide additional maintenance Fire Safety Blankets For a long time mainly used in USA, but USA has highest fire fighter death rate USDA Forest Service concern about entrapment risk with improved 2003 fire shelter, developed with NASA help of energy % of world July 2005 Britishtype Columbia

bans use fatalities of fire shelters to prevent 1.8 lives 1.8 million million lives entrapment already already saved saved by by Nuclear Nuclear nuclear energy! nuclear energy!Energy Fossil fuels electricity 5163 total 300,000 per year 12.3% from 437 plan ts 69.4% Axioms & Approximations Use linear approximations Equation will certainly be valid about an operating point: May or may not work well for large deltas Cost of innovation axiom: (equilibrium condition) Corporations will engage in innovation (adding Features, new models, etc.) until there is no incremental profit PF from doing so Subsume all costs in PF except cost of the crash rate CR P F CR = 0 C R = P F Similar in concept to marginal utility of safety (Spence 1975, Savage 1999) Development Crash Rate approximation Divide development cost Cdev by problem (bug) rate Rd to get cost per development bug Cd Agrees with

with our our experience experience that that hardware hardware To get total feature cost add manufacturingAgrees CF CdRd + M (M0) (M0) Rewrite: Rd (CF M)/Cd has has fewer fewer bugs bugs in in development development than than software software (C (Cdd0) 0) Axioms & Approximations CONTINUED. . . Operational Crash Rate approximation Use the concept of Defect ratio (defect leakage through testing process) Not applicable to component wear/fatigue, but those processes are already well understood Most modern failures are latent design or procedural issues: All software failures, Fukushima reactor, Boeing 787 battery, Deepwater Horizon blowout, both Shuttle losses, etc. Treat procedures like software, i.e. latent design issues (design of the procedure, or design of enforcement) Even fatigue failures become latent design issues, i.e. they should have been caught by inspection / maintenance procedure It follows that the operational failure rate is the development bug rate times D: RO DRd (use consistent units per hour, flight, device, etc.) Substituting for Rd we have R CF ROCdabout /D + M Note: previous observations development

O (Cour F M)D/C d or Note: our previous observations about development rates bug rates Gives a relationship between operational failurebug rate RO and cost of now now apply apply to to operational operational failures failures ~ ~ features CF more expensive development process process lower lower the the crash crash rate rate more more valuable valuable features features more more failures failures more expensive development Now we can apply our equilibrium condition the cost of innovation axiom The Crash Rate Model

First, a profit axiom Economic utility (Value) to users of the given Feature set: VF Seller/producer will set price = VF to maximize profit: PF = VF CF Combine previous axioms & solve for crash rate: Apply the innovation axiom to get cost of crashes (failures): C R = VF CF Use operational crash rate axiom to replace C F : CR = VF (ROCd /D + M) High value, easily produced features encourage more use and more risk taking CR/R O high RO Solve for crash rate: RO (VF M CR )D/Cd Express crash costs as cost per crash C C = VF M RO CC Cd / D High cost per failure (e.g. air or nuclear) conservative & careful use, low RO High development costs lower crash rate Verification, Verification, inspection inspection ,testing, ,testing, analysis analysis & & quality quality control control multiply multiply the the effect effect of of C Cdd even even if if they

they are are cheap cheap Verification, Verification, inspection inspection ,testing, ,testing, High value, easily analysis analysis produced & & quality quality control control features multiply multiply the the effect effect of of C Cdd encourage more even if if they they are are cheap cheap use and more risk even taking high RO VF M RO CC Cd / D High cost per failure (e.g. air or nuclear) conservative & careful use, low RO High developm ent costs lower crash rate

The Crash Rate Model EXPLAINS: 1. How 6-sigma reliability helped Japanese automakers become largest in world First, a profit axiom (low D) -Economic utility to users of the given Feature set: VF Now virtually all cars(Value) will go 200,000 miles - After 20 years of competitive evolution, Japan has lost this advantage Seller/producer will set price = VF to maximize profit: PF = VF CF 2. How Boeing, ATT & IBM dominated with expensive but reliable products (high M & CC) previous axioms & solve for crash rate: Combine 3. Why you may not want to take a fast-tracked drug (Cd too low, D too high?) Apply the innovation axiom to get cost of crash crashes (failures): C R = VF 4. Why your PC and phone crash a lot and are subject to hacker attacks CVF , low everything else) (high F Use operational crash rate axiom to replace C FHigh : CRvalue, = VFeasily (ROproduced Cd /D + M) Solve for crash rate: RO (VF M CR )D/Cd features encourage more use and more risk taking high RO

VF costs M as cost per crash C = C /R Express crash C R O R O CC Cd / D High cost per failure (e.g. air or nuclear) conservative & careful use, low RO High development costs lower crash rate Verification, Verification, inspection inspection ,testing, ,testing, analysis analysis & & quality quality control control multiply multiply the the effect effect of of C Cdd even even if if they they are are cheap cheap VF M RO CC Cd / D The Bad News

Competitive equilibrium The formula assumes operation at the optimal profit point If a company does not operate there, it will be acquired or bankrupted because others have more money By culture, contracting, employee rotation & use of the same management consultants, Government generally operates close to the same point as industry We have seen many administrations pledge to make Government as efficient as industry Data may be unavailable for D during development, V F for govt programs (i.e. profit) The Good News It may be possible to shortcut 20+ years of trial and error and choose a good operating point but we must learn how to react to new data 1979 Honda Civic 1979 Chevy Nova http://www.uniquecarsandparts.com.au/car_spotters_guide_japan_1 979.htm ? http://www.detailshop.com/rides.php Application to Examples VF M RO CC Cd / D Montana speed limit? Perception of greater risk (high RO) from Bad crashes with speeding driver, high C C Greater risk of crash due to other drivers high D (errors) Greater cognitive awareness of all risk factors vs. posted speeds which are considered by drivers to be conservative No-fault auto insurance? Reduced CC to driver, due to better insurance coverage and lack of fault penalties Seat belt and air bag effects? Perceived slightly lower cost CC (damage) from crashes when buckled Relative to incorrectly perceived lower than actual risk/cost of unbuckled driving

Incorrect perception of near-immunity to C C with air bags, most crashes not head on, poor positioning of unbuckled occupants during crash i.e. bags are of almost no value unless buckled up, and most of the protection comes from the belts Application to Examples VF M RO CC Cd / D War on drugs? Add supply-focused enforcement to cost of M VF made higher by erroneous just say no commercials User CC made lower by medical advances and free emergency room ruling War on cancer? D 1 due to lack of a cure (all defects become operational defects) Perceived VF rises as hope for cure persuades people to undertake expensive treatment Air transportation? CC is high and perceived higher (lottery effect, disaster avoidance effect) D is very low due to independent certification & investigation Dependence of VF on RO, value of high risk airline or airplane drops to zero Application to Examples CONTINUED VF M RO CC Cd / D Fire safety blankets? Incorrect perception of low CC of entrapment Nuclear energy? Perceived extraordinarily high CC of failure Low D for same reasons as air transport We will analyze this in the exercise next! Shuttle orbiter? VF is not directly measureable for non-profit space projects perhaps use total

cost? Cd and D are not known from easily accessible public records Large testing costs applied to engines, avionics, tiles with many testing defects corrected high C d/D SRBs and foam were considered mature (low testing?) and many operational defects were ignored (high D) Formula is meant to analyze a change (delta) After 1986, military & commercial dropped presumably V F lower, change in RO from 1/50 to 1/84 (nearly double) In 1986 change from quasi-military crew to civilian (teacher) no re-look at R O compare to WWII bombing runs Between planning and ops there was a 100 to 1 reduction in flight rate how would this affect R O ? Project Manager Exercise: VF M RO CC Cd / D Commercial transport to orbital hotel / tourism facility Goal is to sell tickets at VF = $1 million with recurring M < $750k per passenger Using next generation SpaceX reusable launcher 20x < cost of current $133M for Dragon 7-passenger vehicle 10 passenger reusable transport, dev cost $250M, 2 copies 1 flight a week gives 10 x 50 x 250k = $250M/yr net revenue Passengers sign waiver of liability but this is not expected to hold up in case of vehicle systems failure At-fault accident liability estimated at $1B and no one will insure at reasonable price Hotel & investors insist RO << 1 / 25 years to guarantee viability 1 / 25 years would be 1 / 1250 flights, still 1000x more risky than a 1000 mile auto trip (1 fatal crash / 100M miles) Testing program 10 test flights, revealing 5 major but not fatal problems, giving C d = $500M/5 = $100M You are confident from risk analysis and test results that D < .1 (.1 x 5/10 = 1 problem in 20 flights) $1M .75 .25M 1 RO $1B $100 M / .1 2B 8000 seems like a pretty good number but . . . Crisis: VF M $1M .75

.25M 1 RO RO $1B $100 M / .1 2B 8000 CC Cd / D After RO publication & failure of a competing spacecraft: Oops, thats only 800 flights, with a crash expected every 16 years Ticket sales top out at 100 due to perceived risk, the venture will fail Hedge fund offers to rescue company, alter ticket price to $5M 10 flights / year, revenue of $500M, mostly profit Founder asks you What is risk of hedge fund plan? How much money do you need to meet original goal of 25 year crash interval expectancy? RO $5M .75 4.25M 1 $1B $100M / .1 2B 470 Due to low flight rate and high financial pressure on each flight new plan expects crashes Spend another $100M forevery 5 test 4.7 years flights and system upgrades obtaining D < .08 reduce ticket price $100k, spend

$250k S/W Air, one of the lowest ticket price carriers, is also one of safest per flight onthe inspection & analysis $.875M .75 .125M 1 RO $1B $120 M / .08 2.5B 20000 Caveats: VF M RO CC Cd / D Linear approximation range may be violated in this example Still extremely useful for detecting direction of change and incremental amount of change Static equilibrium equation only Does not consider dynamics (speed), but humans respond remarkably fast, consider aviation inferences All eggs in one basket equation, portfolio (Markowitz) approach unused by NASA since early 80s Determination of D is not an exact science for small sample size projects Infeasible to mount thousands of large missions Use engineering analysis & inference methods along with independent verification [next slide] RO $5M .75 4.25M 1 $1B $100M / .1 2B 470

$.875M .75 .125M 1 RO $1B $120 M / .08 2.5B 20000 Due to low flight rate and high financial pressure on each flight new plan expects crashes Spend another $100M forevery 5 test 4.7 years flights and system upgrades obtaining D < .08 reduce ticket price $100k, spend $250k per flight on inspection & analysis A Methodology Suggestion HOW TO APPLY SMARTS EFFECTIVELY VF M RO CC Cd / D Function similar to Shuttle SE&I office set up after Columbia Re-development as if test program continuing Independe nt Testing Defect Ratio Est.

Ops Mature Reliable System Developer Testing Production Defect Estimate Development Development Defects RePlan Event Low Defects High Cd Project Design High Volume Low-Risk Return Re-Design Opr. Inspection & Monitoring VF M RO CC Cd / D Conclusions: Provides insight into effects such as mission frequency and testing Economic theory (supply curve, more is more difficult) is at odds with Engineering experience (learning curve) Needed to provide rapid adaptation to new technology 20 years is too long to gain experience with current & proposed rates of introduction of new technologies Cheap vehicles need many test flights to lower D Expensive verification is incompatible with the concept cheap relying on CC restricts value of missions VF Alternative (used in the exercise) is to get D very low, taking advantage of the low cost of missions for verification Need a way to account for distribution of severity of defects (often logarithmic, e.g. earthquakes, foam loss?)

Re-development as if test program continuing Independen t Testing Defect Ratio Est. Op s Mature Reliable System Developer Testing Production Defect Estimate Development Development Defects RePlan Event Low Defects High Cd Project Design High Volume Low-Risk Return Re-Design Opr. Inspection & Monitoring Summary of Engineering Effects industrial creative design

crash rate VF M RO CC Cd / D engineering, automation & labor management system engineering and project management test and verificatio n safety engineerin g social engineerin g process efficiency

Recently Viewed Presentations

  • BrainGate Chip - Auburn University

    BrainGate Chip - Auburn University

    Brain-Computer Interface - BrainGate Chip Hillary Grimes III Homework 6 COMP 4640 Brain-Computer Interface (BCI) BCI - direct communication pathway between a brain or brain cell culture and a device (computer) One way BCIs information passes from brain to computer...
  • Scientific Posters George Kantor and Howie Choset http://www.ncsu.edu/project/posters/IndexStart.html

    Scientific Posters George Kantor and Howie Choset http://www.ncsu.edu/project/posters/IndexStart.html

    Scientific Posters George Kantor and Howie Choset http://www.ncsu.edu/project/posters/IndexStart.html http://www.swarthmore.edu/NatSci/cpurrin1/posteradvice.htm
  • Surface Area - Denver School of the Arts

    Surface Area - Denver School of the Arts

    Steps to find Surface Area. Works for Prism, Pyramid or Cylinder. Draw a net and label the dimensions. Calculate the area of each face and base. Add all areas together. Or. If Prism. Find perimeter of base and multiple by...
  • Koalas - api.ning.com

    Koalas - api.ning.com

    El cambio climático es tan evidente, que durante una larga semana la temperatura en Australia, no bajo de los 48º y ocurrió algo nunca antes visto. Los koalas que se ocultan de la gente y que solo se los puede...
  • CS 98/198: Web 2.0 Applications Using Ruby on Rails

    CS 98/198: Web 2.0 Applications Using Ruby on Rails

    Product Owner tries out the user story and then either hits"Accept"or"Reject" Accept, which marks user story as done, or . Reject, which marks story as needing to be Restarted by developer. Follow scrum roles, Product owner signs off
  • HAEI and HAEII course objective #16

    HAEI and HAEII course objective #16

    Assessments to be changed: 6 on-line weekly quizzes (60 pts), student group grade for problem-solving presentation (10 pts), quiz at end of December block (40 pts); non-cumulative quiz at end of January classes (~3rd week) (40 pts) Edit some of...
  • Project 1 - Exposure to UV radiation

    Project 1 - Exposure to UV radiation

    Photo: Helen Mason. Running gear - bright so cars can see you, cool. Photo Credit: John Dudeney. Futuristic space suits may be similar to scuba diving wet suit made from neoprene. Photo / Donna . Coveney, Open Image Gallery. Dava....
  • Nervous System - Western Oregon University

    Nervous System - Western Oregon University

    Peripheral nervous system Nerves, neurons, and sensory organs outside the central nervous system Functions: Sends signals to the CNS Receives and transmits motor signals from the CNS Stimulates effectors Somatic nervous system Motor neurons that control voluntary movements by activating...