Showing posts with label graphs. Show all posts
Showing posts with label graphs. Show all posts

04 February 2010

It seemed like a good idea at the time

My thanks to King Banaian at Hot Air for pointing out the testimony of Mr. Pinto and the attachments therein.

Note finances, borrowing, etc. are not an area of my expertise and I am giving the best shot at this that I can without deep backing in economics, lending markets and so on.  That said I do know how to read graphs, have a good idea of risk assessment from not only my personal life but my prior career in the civilian side of DoD, being a generalist systems analyst in my outlook and having to manage my own finances knowing my body has a very high risk downside.  That and having read a bit about the way the federal government has gotten into the housing system.  And I haven't seen much good before writing this article.  I am not amused.  What follows is lengthy (very, very lengthy) as I take time to explain what I am reading and seeing, and what I draw from that.  YMMV.

Yesterday was the last day for complaints.

In reading a presentation before the Subcommittee on Housing and Community Opportunity of the Financial Services Committee of the House of Representatives by Edward J. Pinto on 08 OCT 2009, it is impossible to not reflect on the intrusive nature of government to change the nature of our economic lives by encouraging 'good' activity via the tax code and other regulations.  Of the interesting items to come up is a chart measuring percentage of FHA foreclosures as a part of outstanding loans from 1951-2009:

Foreclosure Rate Pinto 2009

Ahhh, graphs!  Anyone wanting to do a quick examination of what you can say with graphs can check out my posting on regression towards the mean.  This is a simple two axis chart to just measure the given percentage, which started out as miniscule at 1951 and has skyrocketed in a two-stage fashion: once 1957-1981, where a definite increasing slope can be drawn although not at a great angle for a median, and then from 1981-2009 which is much steeper.  Of course a single less steep median from 1951-2009 can also be put in which would trend below the large spikes in the 1960's and again in the 1970's, then above the line until 2002 or so with the years since then trending above the line.  The smaller time-period break down gets two phases that may or may not be warranted, and the single line gets an overall trend from start to finish.  Mr. Pinto goes on to present papers done previously that start to put in the variables that get the overall trendline and those dynamics tend to play out more in a complex graphic arena, although broken down into 2-axis graphs for simplicity of presentation.  I would suggest that at least ternary graphs need to be invoked multiple times to show multi-variable cost/benefit ratios, but those would take awhile to do and I don't have good graphing software to handle the stuff. Mr. Pinto's main presentation goes from approx. 2005 to 2009 covering the banking bailouts and other problems seen in the home mortgage and financial industries, and then presents historical papers to examine the overall problems and how they developed.

The first of these attachments is by Peter J. Elmer and Steven A. Selig of the FDIC Division of Research and Statistics done as a working paper in 1998 on The Rising Long-Term Trend of Single-Family Mortgage Foreclosure Rates.  It is a fascinating document as it does a multivariate analysis of how individuals and families have approached their finances, risk assessments and investments in response to changes in the tax code.  It is a working paper that is as fascinating for what it doesn't say as for what it does, and I will cover a couple of salient sections of it that examine just how we respond as individuals and families to structural changes in the financial arena are non-obvious and yet readily apparent when you think about it.  As can be seen from the chart of Mr. Pinto, the foreclosure rate in 1997 was still increasing and the FDIC wanted to find out why this was happening.  I will be retyping their text and doing the best job I can of transcribing the work involved.

From early on this paragraph sticks out and emphasis is mine:

All theories of mortgage default stress a key role for homeowner equity, and empirical analysis supports this emphasis.  Since the most direct measure of equity is the loan-to-value ratio (LTV), we expect to observe a positive relationship between LTVs and foreclosure rates, although the relationship may not surface until several years after mortgage origination.  As Figure 3 illustrates, rising LTVs explain several, but not all, aspects of the foreclosure rate trend.  In the early 1950s mortgage lending was remarkably conservative, as witnessed by an average LTV of only 58 percent in 1952.  Rising LTVs throughout the 1950s suggest a transition to modern-era lending practices, when LTVs have averaged over 70 percent.  This transition explains the exceptionally low default rates of the 1950s as well as rising rates in the early 1960s.  Unfortunately, LTV trends fail to track foreclosure rates for two decades after the mid-1960's.  A possible relation reappears in the late 1980s and 1990s, as slowly rising LTVs again follow rising foreclosure rates.  However, this most recent relationship is questionable because of the close relationship between conventional and FHA rate trends, as noted in Figure 1.  That is, since FHA mortgages have had high LTVs for many years, and the FHA patterns in Figure 1 are very similar to conventional patterns, it seems unlikely that rising LTVs are solely responsible for the rising long-term trend in mortgage foreclosure rates.

Elmer Selig 1998 FDIC Fig 1
Elmer Selig 1998 FDIC Fig 3

Yes if you were doing trend analysis using regression towards the mean you would have identified the bi-modal system just like the experts did!  You get one mean from the 'modern-era' transition and then another one that appears in the 80s and 90s.

With that said there is a misdirection in the analysis in that paragraph in examining LTVs.  The analysts put forward that there is a positive relationship between LTVs and foreclosures, so that if the lender covers more of the value of the home then there is a net trend to higher foreclosures.  Do note that while the FHA tracks conventional rates of foreclosures, there is a delta between them as the FHA is directed to cover more of the value of the home, thus having a higher LTV.  Figure 1 and Figure 3, thus, have a relationship between higher FHA LTVs and higher foreclosure rates for FHA loans.  While that difference is miniscule at start in 1951, by the 1970s the delta between foreclosure rates varies between 0.5% to 1.0% higher for the FHA vs conventional financing as seen in Figure 1, and that delta rarely gets below 0.5% higher for the FHA thereafter.  The FHA covers more of the value of home mortgages and assumes risk for doing so via higher foreclosure rates.

Figure 3 lacks the FHA foreclosure rate so as to get a direct 1:1 comparison between their LTV and foreclosure rate.  Going back to Figure 1 and getting the deltas allows us to see that if this had been done there would be a need for some exaggeration of the FHA LTV as it spikes to nearly 2.0% in the mid 60s and over 1.5% in the mid 70s.  However conventional foreclosures are a lagging proxy for FHA foreclosures and is also used as a reference.

What is a 'lagging proxy'?

A 'proxy' is a closely related graph that has high correlation to another graph plotted along the same axes, but having divergence from the related graph.  In using only conventional foreclosures, the authors of the paper have put in a proxy for the FHA foreclosure rate in Figure 3.  There are many reasons to do that, like graph clutter, but they have far busier graphs elsewhere in the presentation.  Another reason is scale exaggeration for a third, related, scale being used in a cross-comparison and that is a good indicator for the use of a lagging proxy in Figure 3.

The 'lagging' part, as seen in Figure 1, is that where FHA is on foreclosures in one year is reflected in overall conventional foreclosures 2 years later.  There is little to no lag in the 1951-60 timeframe, but thereafter a minimal 2 year lag appears, so that when FHA foreclosure rates reach a local high in 1964 the conventional foreclosures reach their local high in 1966.  At the end of the decline at the local trough the FHA reaches it in 1969 and conventional rates reach it in 1972-73.  The next FHA peak in 1973 is seen in 1976 for conventional rates.  There is less correspondence on the weak multi-year trough,  and even a slight reversal in 1979 where the FHA lags conventional foreclosure rates, after that you get a period of 1:1 correspondence of slope until the FHA reaches the beginning of a spike in 1989 followed by conventional rates in 1991.  Overall the conventional foreclosure rate either mimics or lags FHA rates, at a lower percentage level.  Due to the nature of the highs and lows seen in larger periods of the graph where lagging happens, the general concept is to use conventional as a lagging proxy.  That means that FHA loan foreclosures are a LEADING proxy and what you see, today, has a good chance of immediate to 2 year correspondence in the conventional foreclosure rates for a given time period.

These are corresponding graphs, then, not coincidental nor, of necessity, linked via causal factors between one measurement and the other: indeed both respond to outside factors and when economic stress happens the more exposed FHA market reacts with volatility because it does lend out at a higher LTVs as the authors point out.  For foreclosures and economic stress, FHA loans are the 'canary in the coal mine' reacting first and with volatility and signaling a problem that is likely to hit conventional loans either immediately or in the very near future.

Now I've put modern era lending practices in a highlight as it is a shift from conservative, old fashioned S&L style lending for home mortgages, done by small local institutions that refuse to expose themselves to high levels of risk, to less risk averse lending practices starting in the mid-1960s.  The older style of lending meant that you had to have a significant portion of the value of a home covered up front and/or in collateral, plus have a good employment record for one individual in the household.  That is the typical, nuclear family lending practice supported by the way families were formed in America just after WWII.  It is also a shift in affluence away from the extended family, which included grandmothers and grandfathers, and possibly entire households of aunts and uncles, living under one roof so as to economize on living expenses. 

That is the old style 'traditional' American family, with 3 to 5 children as a norm making for a full household.  Even when such families didn't live under one roof, they were close to each other, normally within a few city blocks, so as to have a close-knit reliance system for taking care of children, watching over homes and helping out on joint projects for the families involved.  The suburban expansion to wide open spaces, cleaner air and the ability to commute to work change where Americans were living but that also meant that lenders were not so ready to take up high risk on unproven suburbia.  This family would have mom, dad and 2.5 children and by the early 1960's mom was on 'the pill', and family expansion rate was going down and mom could re-enter the workforce once the children were in school.  The neighborhood S&L was a mainstay of that early suburban life and only started to fade when higher risk lenders, like FHA, got into the market.  That enticed commercial banks to higher risk lending and led to the slow decline of the S&L system.

The result?  What is the result when less local, more distant banking institutions move into a market and assume more risk on loans?  First you get a marginalization of the older lenders who are restricted in their lending practices by statute.  That is why S&Ls sought the easing of regulations as those very same, conservative, regulations were killing them and they were being replaced by other lenders who had more liberal lending practices allowed by statute.  Congress has played a large role in the banking and finance industry by setting up systems that may make sense when they are put down, but end up picking winners and losers in the long run.  When S&Ls had to directly compete against commercial institutions and the federal government, itself, and they had more liberal LTV rates which assumed more risk, these S&Ls were being forced from the business by Congressional fiat.  By the time the regulations were eased, the S&Ls grasped at anything, including very risky loans, and the failure of the S&L system is the result and the taxpayer had to bail them out because Congress was a major cause of the failure.

Remember if you like nice, tight, federal regulations, then you had those in the early 1950's when the FHA could not assume a higher LTV save marginally, and there was a bright line in lending for homes between locally controlled S&Ls and non-locally controlled commercial banks.  All the regulations since then, to liberalize loans so that institutions take on more risk has caused an increase in foreclosure rates and wiped out that most conservative part of the banking industry.  That happened by regulation, not by lack of it.

Fascinating what you can get from a data analysis when you put it in wider context, no?


The next paragraph to look at follows the first:

A second variable that effects homeowner equity is the rate of appreciation in house prices.  High home appreciation expedites the buildup of equity by reducing the current LTV, i.e, loan to current value.  Other variables being equal, high home appreciation is expected to reduce defaults as current LTVs decline and wealth increases.  As shown in Figure 4, to indexes of house appreciation increased in the late 1960s, remained high until the early 1980s, then dropped to much lower levels until the present time.  These trends suggest that house appreciation is especially useful in explaining the relatively low rates in the 1970s and some rise in the early 1980s.  However, the relative stability of appreciation rates through most of the 1980s and 1990s is difficult to reconcile with the continued rising trend in foreclosure rates as well as with the plateau apparent in the mid-1990s.

Elmer Selig 1998 FDIC Fig 4

See?  I told you they had a busy graph!

And there is our old friend the Conventional Foreclosure Rate showing up again, which is a real help with these graphs.

Before modern day lending practices were put in place, a home's value wasn't expected to rise very much over the life of the mortgage.  Taking a look at the early part of the graph from 1954 to 1966 and you see home appreciation rates rarely getting to 0.05% per year, and most of it under 0.02% per year or between 20 and 50 cents of appreciation per $1k per year.  So if you had a home worth $50k it would go up between $20 and $50 per year, more or less, and over the life of a 30 year fixed rate loan you might see home appreciation go up by $2-3k.  Your home would appreciate in value, but not quickly was the expectation coming into the post-war era and a home was a shelter and moderate investment that you wanted to get paid off so you could own it fully, free and clear.  That changed during the late 1960's, and as foreclosure rates started to drop home appreciate rates started to climb.  For this graph note that the CPI-Shelter is a lagging indicator, save for a few years, of the NAR Median Sales Price and that figures: when you pick up a more expensive home it increases your cost of living as part of the CPI for shelter.

If we take the analysis that the transition to modern era lending is in the mid to late 1960s then the CPI reflects that change as the amount you can take out for a mortgage increases (because of the larger LTV) and as foreclosed homes come back to the market we see the appreciation of homes increasing.  When Median Sales Price appreciation for homes gets put in we see them leading the conventional foreclosure rate by a couple of years.  Just like the FHA foreclosure rate also leads the conventional foreclosure rate.  That period from 1970 to 1980 is a good indicator of a leading proxy, but it falls completely apart with the decline in home appreciation between 1980 and 1982.  At that point foreclosures continue their upwards path (all the way to the current day) while home appreciation rates remain steady within a band from 0.02% to 0.07% per year.  Something is going on to de-couple foreclosure rates from home appreciation rates and it begs the question: how can you have steadily increasing foreclosure rates and steady increase in home value?

I would expect that is a multi-part answer as it would be difficult to put a finger on any one thing that is causing that.

Some factors I would expect?

The Community Reinvestment Act starting to ramp up its output in the early 1990's, when we start to see Adjustable Rate Mortgages and 'no money down' mortgages, which are essentially maximum risk LTV mortgages.  Still those were not a common part of the market early in that plateau and would only be a contributing part in foreclosure rates increasing, although 'turning homes' for quick sale and cash would also see a nominal appreciation level that was unsustainable.  When the term NINJA loan was created to cover those with No Income No Job or Assets getting loans, then the end of the speculative and high LTV era was coming to a close.  They were a causative symptom, in that they showed up due to legislative factors and regulation changes in the 1990s, which finally burst the bubble.

Another thing that might hit this is a saturation in the expectation of home prices with an expectation of steady increase in home value far above that conservative 1950's rate.  Once an area gets a relative saturation level for prices, and that varies widely by geography and income factors, there would be a move to build new tracts of similar or slightly better housing at those new price points.  New builds expand the market and put a cap on appreciation as new unit availability keeps the appreciation of other, older units down.  There are some indications of that in areas where I lived, including the financially strapped Western New York region during the late 1980's.  Even with the middle of the 'Rust Belt' still lingering, new subdivisions were going into suburbs at a rate that had not been seen since the mid-1960s.  Moving to Northern Virginia and examining the home market in the area, it was easy to discern much higher levels of building from the mid-1980's then a lull in the early 1990s and then a gangbusters expansion of subdivisions from 1994-2006.  As all units saw an increased and unsustainable appreciation the indications of a housing market bubble became clear and it was only a matter of time before it burst, sending values tumbling.  So capping of appreciation via new builds and expanding the market also plays a factor in this.


I am giving a section on Trigger Events that cause foreclosure to start short shrift. I am a bit more interested in the overall market analysis of how the environment for increasing foreclosures comes about and less on the actual events that cause households to go into foreclosure.


Changes in risk assessment, however, is part and parcel of foreclosures and Americans have had to change their risk map to cover the changes in terrain due to legislation.  That is covered under Household Risk Posture for foreclosures, but are an indicator of expectation changes in the market, also.  The type and amount of risk you take on changes when the cost/benefit ratios are altered by regulation, and those regulations can be far outside the housing market.  Here we can see that a set of legislation meant to 'do good' changes our perception of risk:

Compounding the problem of lower savings rates is the fact that an increasing proportion of savings are being held in relatively illiquid forms, such as 401(k) and IRA types of retirement savings plans.  Although one can make "hardship" withdrawals from a 401(k) plan to protect a home from foreclosure, the penalties are severe.  The IRS requires the plan sponsor, or trustee, to withhold the estimated income tax on the withdrawn amount plus a penalty equal to 10 percent of the withdrawn amount.  Thus, for example, a borrower who needs $1,000 to meet a mortgage obligation, and pays a 20 percent tax rate, would have to withdrew $1,428.57 to receive the amount needed.  Hence, this type of tax-sheltered saving, while ideal for retirement, is not effective as a safety net for adverse shocks to income.

A bit further on is the dovetail to this:

Not surprisingly, the vast majority (approximately 65 percent) of the increase in debt has been in the form of mortgage debt, which comports with the rising LTVs noted in Figure 3.  Of course, the tax deductibility of the mortgage interest stimulates individuals to rely on mortgage debt as a primary form of leverage.  But how can tax incentives be motivating higher leverage if they have been in place for many years?  Engen and Gale (1997) suggest a fresh perspective on this issue that provides an economic rationale for recent trends in increased mortgage borrowings.   That is, their study finds that increased savings in 401(k) plans are associated with increased mortgage debt and a reduction in home equity.  In essence, the financial advantages of 401(k) plans may be causing individuals to substitute savings in 401(k) plans for savings in home equity, thereby causing both mortgage leverage and the likelihood of default to increase.

This is a 'No Shit Sherlock' moment.

If you have an income and have a finite set of ways to divide it and you can invest money in a relatively well protected from any disaster financial plan for retirement or into a home that you could lose during an economic downturn, in which, exactly, will you place your money?  And if you can use your mortgage debt as a means to leverage the remaining income, would you not want to get the most home for the money with the knowledge that it may be a transitory home in which you will get little to nothing from it?  If you go bankrupt your IRA tends to be untouched, in most cases.  Your home flies off the table faster than you can sneeze.  And if you see that there is a steady appreciation of home values and are willing to take the subsidized risk of your income loss for a few years, you can turn that home and perhaps even make a small profit.  The concept of 'home equity', where you build up the amount invested in your home goes out the window as it is unsafe in the modern lending era due to the high LTV and the small stake you have in a high cost home.  In such a market when you are a new entrant and you have two ways to defer or mitigate taxes, one which builds up relatively safe equity and the other that doesn't, which do you put your money into?

The shift to liberalizing lending practices and the legislation to back it, means that lending institutions take on the bulk of the risk for home mortgage lending and the borrower has taken little to none.  The tax supported 'good' of a home mortgage interest deduction means you don't bear the full cost of what it takes to pay off a mortgage and the perceived risk is then lessened due to the deduction.  By being able to put in tax deferred savings for retirement, you gain future benefit at relatively low up-front cost compared to the discounted higher cost of a home mortgage.  And if you can take out a second loan based on the home mortgage, then you can begin to maximize expenditures as you expect the rising value of the home to cover your mortgage obligations while you pay set monthly amounts on loans for big ticket items, thus taking cash out of the household income flow depriving the rest of the areas of household spending of that amount.  By seeing a continuation in home appreciation from a high base, an individual may feel 'empowered' to spending far more than their income allows via debt vehicles.

These two putative 'goods', saving for retirement and home mortgage interest deduction, are done via the tax code, not the lending code, and are overlooked as causal points for foreclosures.  Even worse is that they change risk factoring for individuals and families in ways that are not anticipated when either of them were passed, as the assumption was that all other economic factors were going to remain static in the lending arena.  Those in the lending legislative areas did not examine how risk changes in the tax code changed perception of acceptable risk.  Worse still is that there is an unspoken assumption that banks can and should take up the entire risk of your home.  The banks responded with lending packages that put off the building up of equity for years, because the people taking such loans were not good risk candidates and at least the bank wouldn't lose much money if any single mortgage holder went belly-up.

Is it any wonder we got a housing bubble from this?


Now remember that as we head into the next paragraph as it has implications for what is going on the past few months:

In addition to savings, insurance provides individuals with a financial tool for guarding against the ill effects of unexpected problems.  In this regard, Figure 10 emphasizes that a significant portion of the population is not covered by health insurance and that this percentage has increased by more than one-third since the data were first reported for 1978.  Moreover, the statistic understates the magnitude of the problem by including population segments that are covered by health insurance in their entirety such as the military and senior citizens covered by Medicare.  Thus, the increase in the portion of the population without health insurance constitutes a significant increase in the overall risk profile of households.

Elmer Selig 1998 FDIC Fig 10

Actually including the number of people covered including the military and those on Medicare is a proper representation of those covered because they are covered.  If it is meant that they are not covered by private health insurance, there is a point to be made, but we are the ones who set up the system we have and must live with its effects.

We also subsidize health insurance via the tax code.  We have business write-offs for a portion of the expense for such packages provided for employees, employees can write off a portion of their cost via itemized deductions.  So why are fewer people getting health insurance?  It is yet another cost/benefit ratio, and when the cost of insurance outweighs the benefit of having it or providing it, then the risk for not having it outweighs the cost of having it.  And those deductions for health care have gone hand in hand with increasing numbers of big ticket lawsuits for insurance carriers and doctors, both of which change their risk stance by increasing costs: premiums from insurance carriers and for doctors passing along the  cost of malpractice insurance to the patients.

So now put in three variables for a household: IRA savings, home equity and mortgage deductions, and health insurance.  If we have been financing our IRA contributions via high LTV mortgages with write-offs on the interest, then the cost of health insurance must be factored in to determine the priorities of a household.  Which is more important, carrying health insurance or paying down your mortgage?  Or investing in your IRA?  The reason I put forth that this thing morphs into a ternary graph is because the household finances are finite and multivariate.  After the cost of food, clothing and paying the utility bills, come keeping a roof over your head, keeping yourself well and investing for your future.  Each of those other variables have their own priority and cost expectations, and with a generally set income divided in a number of ways, you only get finite sub-portions left for each of those portions, and we prioritize those based on cost/benefit trade-offs.

'Mandating' health care then imposes another large chunk of finite cash expenditures to be used up on that coverage.  So what won't you do if you have that mandated?  And the costs will rise as that has been the expectation with medical care ever since legislation was passed to help write off part of its cost via itemized deductions.  And as we have gotten so easy on risk with homes, and that is now ending, you can't bundle the cost into expected price appreciation there.  Nor can you withdraw money from your IRA to cover medical expenses, save for catastrophic ones where surviving is more important than how much you will have to pay in the way of taxes.  And if it is a mandatory part of the budget, either via monthly payments or paying a 'penalty' for not buying it, those are non-zero sums and must be factored into the residual cost/benefit analysis.

The paper looks at the trends of bankruptcy and foreclosure rates, and they are intimately linked as both are related to financial difficulty.  What comes out of that examination, however, is not the sort of analysis I expect from the FDIC:

The coincident rise in mortgage default and personal bankruptcy rates is also intriguing from the standpoint of society's attitudes toward leverage and financial risk.  That is, the trends are consistent with the notion that households have increased their risk posture by opting for greater leverage and lower net savings.  Of course, these trends also reflect the willingness of lenders to take on greater risk by increasing the availability of credit to highly leveraged households.  Lenders and borrowers must both embrace these changing attitudes towards risk before an increase in risk can be contracted at market prices.

A fascinating paragraph in the tone it takes towards those that get involved in the mortgage contract market.  It is at one stroke passive and authoritarian.  It is passive in that it describes a changing risk market in the abstract divorcing the causes of those changes from the changes, themselves.  And the authoritarian tone comes with the last paragraph which has the feel of 'thou shalt deal with this changed market in which you have no power to stop the changes' sort of deal.  The unspoken assumption is that the changes are wholly divorced from the causes and the market, full of individuals and companies making contracts, must adjust to the landscape being changed by these forces and accept that changed landscape without asking questions about why it changed.

After they have examined the role of government in IRAs and home mortgage deduction, the conclusion that society is changing its stance on risk is not something that can be supported.  The risk landscape is redone multiple times via legislation from 1951 onwards by government without any check from society on it.  Social activists have been well neigh wanting increased risk exposure to the middle class to gain 'benefits' for social causes often masked as being 'for the poor' or 'for the sick' or 'for the children', not for society as a whole.  The incrementalist approach has been to take existing federal programs, FHA say, and then start changing its risk acceptance via law and regulation, not via being implored from society, as a whole.  Something like the mortgage interest deduction is purported to be 'for the middle class' or 'for the home owner' but is a change in what is expected in the way of risk and benefits from home ownership.  Yet the legislation for those things did not start big, but small, and were grown over time as functions of government regulation.

In the Structural Change section the paper does begin to examine this, but consider that the paper accepts that these are changes directed by society when, in fact, they are coming to appease 'social activists' who want to liberalize banking to 'the poor' or those who face 'discrimination' due to ethnicity.  To get to that point, however, takes original changes to the structure of the lending system:

During the 1950s and 1960s most single-family mortgages were originated by "traditional" lenders, primarily savings and loan associations and mutual savings banks.  In addition, mortgage bankers served as correspondents for insurance companies that invested in mortgages and for thrifts in capital-surplus areas, such as some cities in the Northeast.  These "traditional" lenders performed all or most of the mortgage lending functions, including mortgage origination, servicing, portfolio management, and investments in the mortgages.  They were headquartered in the local markets, where they originated loans and typically had other business relationships with the mortgage borrowers.

That is, perhaps, the best summation of the traditional mortgage lending system I have ever run across.  Very clean and succinct.  It is this traditional system that would be targeted by banking and lending legislation:

The advent of mortgage securitization in the 1970s changed the borrower/lender relationship breaking apart the various functions that had been performed by the banks and thrifts.  In particular, it becomes much less common for the same organization to both originate a mortgage and retain it as a portfolio investment.  Lenders with traditional ties to the borrowers were replaced by national service organizations with no tie to the borrower apart from the mortgage and with servicing policies based on national rather than local standards.

Here, again, the passive voice with securitization.  Mortgage securitization did not 'just happen' and was not due to some natural happening or to the Gods of Financing saying 'mortgages shall now be securitized'.  Quite the opposite, it was intentional federal policy.  Mortgage securitization started with the Department of Housing and Urban Development in 1970, and the creation of the Government National Mortgage Association (Ginnie MAE).  You know, the folks who gather up bad loans and turn them into AAA government backed investment vehicles?  Risk pools for the FHA?  Heard of those things in the recent home mortgage 'crisis'?  Started as a swell service to help disperse risk and, instead, ended up saying that high risk stuff is low risk, which is how incrementalism works from 'activists' who want policy changes bit by bit.  First you get some conservative regulation words together, create a bureaucracy, back it with the federal government, spin it off as a GSE, regulate that, change the regulations so that more risk is added to the system to appease 'social activists' who want more money flowing to those who can't pay it back, which changes the entire risk structure of lending.

That is how the commercial banks started getting into the mortgage business as these risk pools would get US GNMA certification as being absolutely, positively AAA in safety.  Goes from a swell program meant to 'do good' to one that injects lots of risk into the system and gets regulators to back that up by threatening to audit banks that don't accept the risk guarantee.  And because this is pooling risk, and going through the right federal paperwork to do so, the smaller S&Ls and thrifts were stuck out in the cold trying to expand their businesses against commercial banks with huge monetary backing and being able to pass the federal sniff test.  Really, how can you compete against a huge bank willing to take on large local risk with 'social activists' pounding on your door and federal regulators telling you that you must lend to people who have high risk?  The S&Ls and thrifts were not designed to utilize high risk vehicles, is it any wonder they crashed and burned in the 1980s?

By creating a national market for local loans the local lenders who knew conservative good risks got priced out of the market by banks able to take on higher risks, locally, and willing to undercut local lenders with deals that those local portfolios couldn't match.  Throw in some 'social activists' and you get a lovely agenda to wipe out the local, community supported banking system and replace it with a national one that can be regulated from DC.  Not regulated all that well because DC can't figure out local markets and takes a bludgeon of national markets in a One Size Fits All, Fits None Well approach.  The creation of GNMA has had consequences in the S&L and thrift area, namely stifling and then undercutting them so they needed a bailout, and it has served as a means of social engineering by Congress to inject risk into the lending system so as to get money out to politically preferred groups who have poor risk profiles.

As the charts demonstrate there was pre-1970 volatility in foreclosures, but after 1970 there is volatility in home appreciation.  By the 1980s and 1990s foreclosure rates become decoupled from home appreciation rates, indicating that the market is making far riskier loans at greater amounts and not letting the market set the balance between appreciation and foreclosure in a low risk environment.  Something is changing that risk profile to allow this to happen.  Mind you this is examining a paper that is bundled as an attachment in 2009 to Congressional testimony and the paper, itself, was put out in 1998.  My analysis is covering the risk time-frame of the attachment and putting it into perspective and a perspective that is not that out of line with the known history at 1998.

The paper examines what happens when non-local lenders get involved in the market: loans become less flexible, there is less forbearance from lenders who are not in the community on repayment with delinquent borrowers and less ability to offer adjusted repayment schedules if an individual temporarily hits hard times.  Without that local knowledge the borrower is treated as a statistic or set of statistics and not as a person living a life.  That is part of the increase in foreclosures, to be sure: traditional lenders who ensured that borrowers had a larger stake in their homes wanted to ensure that such borrowers did not lose that stake via financial hardship, while via securitized loans distant banking institutions cared less about the individual borrower and more about repayment schedules.  When risk is added to the market via that securitization process, the lenders make loans on property at higher values due to the changes in risk assessment via securitization and are enticed via regulators to increase loan size while decreasing collateral and initial borrower stake in the property.


A section on Empirical Results I will largely bypass.  It is an examination of traditional foreclosure rate models which the authors then examine to see if there can be new additions to the model to better examine how foreclosure rates have changed in regards to steady appreciation.  To that model they add: liability-to-assets ratio, business failure rate, the share of mortgage owned by someone other than the owner.  From my scientific background what I am seeing is a model that once worked trying to be adjusted or 'fixed' to take into consideration phenomena that are not predicted by the previous model.  What this points to is not the need for adjusting an old model, particularly when the new variables now outnumber the remaining old ones, but for an entirely new conceptual approach to what is being modeled.  When new empirical data arrives that is in discord with an old model and it has to be patched to try and follow what is coming in via the data, you are not on firm ground for a sound theoretical basis of what you are explaining.  Something similar happened with planetary observation and the retrograde motion of planets in the night sky over a series of observations was breaking the model of 'crystal spheres' that the planets were a part of.  Epicycles and sub-spheres were added and even allowed you to keep track of planetary motion, although the math was daunting, and it took a new conceptual framework of planetary motion to explain the phenomena via differential orbital mechanics.  That is what I get in the overview of that section: a need to thoroughly revamp the nature of the model so that the events being witnessed are based upon the natural outgrowth of causative events in a wider context.  Here the individual foreclosure happens in a highly volatile atmosphere that has many factors out of control of the individual and examining rates must take that wider atmosphere into account, and not just the personal space around an individual. 

But that is only my quick glance at the topic.


The conclusion presented is interesting:

The rising long-term trend in foreclosure rates is at least partially explained by a variety of variables.  Although traditional determinants of default, notably house appreciation and LTV explain some portion of the long-term trend, they appear to stop short of explaining the more recent, unsettling, rising trend.  In an effort to explain the remaining portion of the trend, we have explored the notion that the incidence of shocks to individual lifestyles or "trigger events", such as divorce have increased.  A related, but distinct, hypothesis is that the risk posture of individuals has increased, especially as individuals increasingly leverage their homes as part of a broader strategy of managing their overall wealth portfolio.  Although evidence exists supporting both hypotheses, the risk posture hypothesis appears more consistent with a variety of disparate incentives and trends relating to household financial management.

And why are households changing their financial management?

The authors have danced around that unspoken question throughout the entire paper.  That households have changed their risk appreciation to manage household finance ignores the drivers of what is changing the perception of risk.  Those driving factors, such as increased risk insertion to portions of the market via FHA lending practices and having those lending practices supported by GNMA post-1970, allows for the insertion of political risk assessment to please sub-sections of society to be injected directly into the mortgage system.  When regulations expand that to include commercial lending practices, the entire market has risk added to it NOT from the ground-upwards, but from the top-downwards via mandates from DC.

Owning a home has, traditionally, had an absolute value attached to it: when you pay off the loan you are free and clear of all outside forces telling you what to do with your risk management.  Getting rid of the mortgage meant that you now had a broader set of funds available for monthly budgeting and you had a valuable asset that provides you shelter from the elements.  If you retired and moved with selling your home, you had a small appreciation in value of the home which would help cover the purchase of a new home, moving and even (with a little work) leave some liquid assets over for investing.  If you moved and rented your home or sold it intra-family, you had a good source of income for many years and still had the option of selling the home, and you would be very interested in having good renters take care of it or be very glad that members of your family appreciated your home enough to pay you for it.  Owning a home was never meant to be an easy thing to do as it represents a huge investment in time, energy and working capital that you removed from your daily life so as to build equity in the home.

With changes in risk assessment and mandated lending practices enforced by regulators, banks became leery of lending such large amounts and changed the lending vehicles to ensure that most of the up-front payments for the first few years were on interest and very little on principle, so that building equity did not start immediately, but was delayed for years.  That puts in a buffer for a bank to make money in a risky environment when told to loan to those that do not meet old style financial criteria as seen before the risk injection period in the 1950s.  That era of the 1950s had artifacts to it that set down how we expected the economy to run with respect to mortgages:

  • Mortgages required a high up-front investment, often over 30%, to demonstrate that one could, indeed, save the money necessary for a home which indicated one had the funds to actually care for and run a home,
  • Mortgages required  a steady work ethic that rewarded long-term employment and put a high value on just being able to 'do a job' to sustain oneself and one's family,
  • Mortgages were lent by  local institutions and, most likely, you also had savings, checking, and other accounts with that institution so as to form a working relationship for wealth management that extended beyond the home to one's daily life,
  • Mortgages stayed locally with the lending institution so it would serve as a source of building up the local community through prudent lending packages that ensured financial stability of those involved in those loans.

The result was low appreciation rates, low foreclosure rates and the homeowner having a high equity stake in their home making it a valued asset, not a commodity.  This formed tightly knit communities that were financially risk averse but pro-growth, as good economic times were required to safeguard the equity in a home and a downturn would diminish that stake via foreclosure and having to pay off the lender, first.

Every change in regulations have removed those stable points in lending practices and the S&L and thrift failures of the 1980s is a result of undermining that model from government regulation that changed the risk assessment system and opened up the commoditization of the home market as the market was nationalized.  That meant that aggrieved sub-sections of society working through 'community activists' changed conditions nationally going through DC, not locally going through State and local governments.  The power of a federal audit to freeze a banking institution is not to be underestimated, and when such threats are utilized to change the risk atmosphere the banks are painted as villains for wanting a low risk lending environment. 

Instead the entire nation gets a higher risk environment that sees the commoditization of homes, lenders making loans to those without the financial backing to pay back a loan, rising home prices that stimulate price-point home construction, and the reduction in equity in homes held by borrowers.  The result is a foreclosure rate that steadily increases and people willing, and able, to walk away from a mortgage by mailing their house keys to the bank.  Going after those who cannot even pay a mortgage and who have poor work habits and no assets is not a good choice for a bank as it has a low rate of return for the legal investment.  And when you try to foreclose on homes, 'social activists' want those who are unable or unwilling to pay for them to stay in those homes and castigate the banks as the evil ones who 'forced' people to walk in the door, ask for such loans and put their names on the bottom line to get them.

By not looking at the causative factors of the changing risk environment, the authors of this paper (which is only one attachment of many on a larger submission to Congress, and I haven't even gotten to the annex of this paper!) hit a roadblock from the perspective of the FDIC.  Why?

First off is the FDIC is the Federal Deposit Insurance Corporation, which guarantees savings that individuals put into FDIC covered banks.  They are interested in foreclosure rates as it may point to a larger banking and financial system problem that will negatively impact the FDIC.  Thus the interest on foreclosures and not on the environment causing them in wider context.  By concentrating on why individuals go into foreclosures, and only giving a light overall market change analysis, the FDIC is concerned with its major topic - individual savings accounts and the institutions holding them.  In isolation that is all well and good, but misses the dynamics of the previous 47 years and the incremental way the market has changed due to legislation and regulatory input.

Secondly, and it is a problem in the federal government, there is an attitude of 'thou shalt not speak ill of other federal agencies'.  I was a member of the federal bureaucracy in DoD and ran across that unwritten rule numerous times, which required the blandifying of citations of problems in other agencies or glossing over them so as not to indicate which agency was at fault.  Being unable to speak out about the FHA, GNMA and other institutions in any save the most remote and third-hand ways, the authors are practicing a job survival skill: do not tread upon another agency's territory.  Turf wars are bad enough with Congressionally mandated programs, don't be stirring up any on your own is the attitude of the bureaucracy.  I sympathize!  Been there, done that, got the t-shirt.  It also means that one's analysis cannot be in-depth, far ranging and investigate fundamental changes to one's area or turf due to changes in the related turf around you.  You dare not even walk up to the line and say it, but stay a good few feet from it and whisper just barely about it.

Third is that the Federal Reserve has that turf.  Or should have it, at least.  But the Federal Reserve is yet another federally regulated entity with its own turf and turf protection schema and has its own agenda system.  Someone should have been speaking out when Fannie and Freddie were lobbying for injecting more risk into the lending system, but the regulators did not want to speak ill of GSEs full of the friends of politicians running them.  Speaking out about the patronage system that goes with the intrusion of the federal government in lending is a sure way to end your term inside the Federal Reserve, FDIC, GNMA or any other institution related to mortgages and finances.  CYA and job protection are Job #1, and 'thou shalt not speak ill of other federal agencies' is wrapped up in that.  These are, perhaps, not the wisest choices for people to oversee each other.  And, no, 'whistleblower' protection laws do not protect you overmuch once the shit hits the fan, and your job is made a living hell because you are blowing the whistle.  Who would hire you after you blew the whistle?  It is the right thing to do, of course, but also puts your long term livelihood at risk, no matter what the legal protections are.

 

As it is these authors have presented enough facts and give enough outlay of the problem so that it starts to define itself, even when they can't speak about it.

17 April 2009

Ternary graphs and what they do for you

Yes, yet another in the series of absolutely esoteric concepts that may actually play a role in your life!

You have to love this stuff, no?

Ternary graphs are graphs that utilize three major axes and can then have additional correlated lines added in to follow other information. Just so you get an idea, a basic ternary piece of graph paper looks like this:

Ternary Graph Paper

Today it is ternary graphing which we used in geology class to help classify the mineral content of rocks that had three primary mineral types so that you could identify the melting temperature of that rock based on mineral composition. Yes, pure esoterica! The good folks at Tulane have an example of this at their Petrology course, and it gives you a nice example of the multiple mineral types and differing melt temperatures associated with them. First would be a 3-D representation of crystallization temperatures utilizing the different percentages of minerals:

ternfig1

Now that just looks horrible! But, if you project it onto ternary graph paper, you get this:

ternfig2

Note that the crystallization temperatures now become boundary curves, so you can start to figure out what crystals are forming when and how long they have to develop. This is great stuff!

In geography you can use it to plot things like changes in neighborhood status due to demogrpahics, as seen at the Geography Field Work site looking at the impact of gentrification on a neighborhood.

TriangularGraph

How does this have applicability to the real world beyond the rocks beneath your feet?

As if those weren't important...

The reason it has applicability is via the other great ternary problem that crops up in all sorts of places. The first great example of the exclusionary ternary problem rule is NASA that started to show up in the 1980's, and as a result its fallout is described like this (Source: Space.com, 13 MAR 2000) and I will highlight the words that are critical:

WASHINGTON - A former NASA manager issued a critical report Monday of the agency's "faster, better, cheaper" approach that has pushed the agency's engineers and scientists to crank out more frequent, low-cost and stripped-down missions since the early 1990s.

The report was written by Tony Spear, who worked on several missions and headed up much of the Pathfinder mission that successfully air-bagged its way onto the Red Planet in July 1997 on a relatively modest budget.

[..]

"As with any major human endeavor, there have been successes and failures in the stress and strain of venturing onto new ground, trying new things, taking risk to gain significant return," the report said.

However, "failing due to mistakes is not tolerable," it concluded, particularly when contrasted to failures occurring due to use of exotic technology or a spacecraft that encounters a strange, unknown environment.

In my mind this is the 3-axis exclusionary rule because that is what it came down to during this period. The quip about NASA projects was: faster, better, cheaper... choose two out of three.

Ahhh... now you begin to see that if you put faster on one axis, better on another and cheaper on a third you can start to see what the cost trade-offs are on a project. Well, that was the idea, in reality you can't do that as getting something that is, say, faster and cheaper ends up meaning, engineering-wise, that you are low-balling the work, fast-tracking it and thus don't have time to invest in better technology. Which means you can get a known, proven, low cost sensor platform out the door fast and a low cost. All well and good.

Now if you need better sensors you have a trade-off: you can try to push for faster design work, which costs a hell of a lot, or go for slower design work at lower cost. What you can't get is a new sensor on a new platform done cheaply and quickly. If you are used to thinking in the 3-axis graph you would think that these would not be mutually exclusive and offer trade-offs between them. The world does not work like a neat ternary graph for new engineering designs.

An old design, however (not better) can be faster and cheaper. A new design is one or the other, not both.

If you can afford a long time to delivery, then you can get it cheaper and better. A faster track means one or the other.

Do you see how this works?

The ternary graph is very good for plotting mutually dependant parts of a system, but when each part has individual parameters that act on their own, you can't plot them on the graph as a change in one of the independent variables has a non-dependant variable change in ways that are outside the plotting realms.

Three interdependent variables are great for ternary graphs, and when a system can be found that has something going on like that, it allows for good interpolation of what happens if any single variable changes: you are looking at dependencies which reflect each other and, so, are known as part of a system.

So with two different ways of looking at three-variable problems, what else can this apply to?

Interestingly enough I think a good case can be made for health care!

Really there are three major determining factors that we all want from health care, and the debate about them really is quite interesting. Let me put them down as I see them:

1) Cost - Everyone wants 'affordable' health care. Well price varies along with many things, but affordability is a basic concept to deal with. So Cost is a definite part of the problem.

2) Availability - This, too, is necessary. Everyone wants there to be enough providers and such, and this turns out to be a major concern witnessing how other systems that are State run or supported soon start to make it less available. So Availability is part of the problem.

3) Quality - Good quality health care has been a mandatory part of the American system and we work very hard to sue the dickens off of poor providers of health care. One mistake and you're out. So this is part of the problem, too, as it is an expectation of the system.

Now are these interdependent, like crystallization points for minerals or how changes in immigration and gentrification and services are interlinked? Or are there independent variables that change in ways not easily told by these three things that everyone wants, which is the NASA concept?

The US system has two major periods that typify it: 1) immediate post-WWII era (1946-60), 2) increased presence of health insurance as a major feature of working life (1961-present).

Era 1 is typified by high availability, good quality and moderate cost. There were very few complaints about the health care system coming into the baby boom era when all the babies were being born and families were expanding at a fast clip. What had once been a wartime benefit to encourage older workers to get to work in the factories during the war (health insurance) continued its subsidized way beyond that necessity, but hadn't shown up as part of the labor/management landscape.

Era 2 is typified by increasing utilization of health insurance subsidies as a negotiating lever between labor and management and saw a swelling of rolls for families getting health insurance. Plus the Boomers would start to appear in the equation, causing an increase in the number of people getting health insurance due to the demographic change they represent. This era is typified by high availability (no one is turned away), high quality and moderate to increasingly high cost.

Just as a rough 'hand-waving' concept, US health care is demonstrating that increased utilization with subsidies added in will increase quality of care, but also increase cost.

This points to an interdependence where cost goes up with higher quality and larger subsidized utilization, and is lower with somewhat lower quality and less subsidization. As subsidization is part of what changes the system, it must be examined.

The UK was in similar circumstances post-WWII, but went from an Era 1 to a different scenario, call it Era 3.

Era 3 is typified by State control of the system, universal availability for citizens and higher taxation. The result for the UK is high to moderate and declining availability of health care services, good to low moderate quality, and low cost. The universal experiment there is pointing to a relative decline in availability (rationing) and a slow lowering of quality standards. Part of the reason for that lowering is the ever increasing dependence of the UK on foreign born doctors due to British citizens not seeing medical practice as a good occupation.

Canada is in a similar boat, most likely due to the similarity of government systems, and is seeing increased rationing (low availability) for what people in the US would consider 'standard' tests and procedures.

Germany is a different case, and gets its own designation as Era 4. It is typified by a low productivity economy, high taxation and mandatory subscription to health care services unless you earn enough to opt-out. Additionally you lose much of the patient choice of physician in the German systems (save at the high end) and this causes patients to 'hop' from available doctors quickly amongst queues. For the amounts paid into the system, Germans are satisfied by their system, but are seeing different problems show up.

Era 4 is typified by high taxation, low productivity, non-choice of physicians (as we know it in the US, but similar to the HMO model for ability to switch due to wait length), and near universal availability of health care. In Germany the cost is hidden in the tax system and doctors only bill directly if you are part of a non-State plan. Thus there is no indication of cost valuation available in the German system. This is a major factor that then distorts the examination of systems, but there are telling points that Germany is moving away from the full State system and encouraging competition and alternative plans to the State one. One does not do that if you are getting good cost valuation, and as the government has so much of the burden and it is trying to get competition, the Cost portion must be reaching unacceptable levels for the government. It does have high medical availability and moderate to high quality (although Germany doesn't do a good job measuring this), and attempts to ensure that everyone has options while maintaining a minimum standard of health care for everyone. But the lack of cost metrics, the low productivity of individuals and high taxes for such a system, are indicators that while the State can maintain a minimum it cannot guarantee good valuation for the cost it has to pay. In essence the government is the purchaser of last resort for the poor and that cost isn't something it wants to continue on with.

There are non-dependent problems that show up in health care between the major variables, which means it is NOT the easy ternary graph... uh-oh. That leaves us with the NASA model.

How would that work?

Cost, Availability, Quality: choose 2 out of 3.

In the case of Germany, UK, Canada and other Nations we can see that if you get low cost you can have available but moderate to low quality health care (UK) or low availability but moderate quality health care (Canada). Or you can get high availability and moderate to high quality, but the cost becomes unacceptable to the payer (Germany).

There are, of course, other models, like the failed Soviet and Cuban ones, that have universal availability, low cost and low quality, but that is unlikely to be acceptable to Americans.

The non-representative factors in the Cost, Availability and Quality model is demographics and type of society the US has in comparison to Europe or other parts of the world. The US has a high productivity society, an expanding population and increasing standard of living for the entire society. Those European nations that have full or major State involvement in health care are now typified by stagnant productivity, contracting population and a standard of living that is only slowly increasing due to general penetration of technology, but the non-use of such technology at the workplace for increasing productivity. Thus to get a social 'good' productivity of the economy to support that 'good' is sacrificed: no one feels the need to achieve greater income to get better care or any of the other State-mandated offerings of social welfare.

As a tool the ternary graph offers many opportunities to examine systems with high degrees of interdependence and also to weed out those that have other factors at play. It is also good to examine changes in such things as demographics where a two axis graph wouldn't show up trends clearly, but a three axis one does. Plus by projecting a fourth set of boundary layers, one can learn how changes in the interdependent variables effect the overall system.

Luckily the NASA system also has its limits, as demonstrated by health care, but its utility does demonstrate that the 'lock-out' of one factor by the other two can show up in other systems that are highly complex. That factor is independent of the areas sought to be measured, thus changing them asymmetrically when that factor is impacted. So even with extremely complex systems, like health care, you can work to isolate three factors that can tell you what the repercussions of the system are even if there is no great 1:1 co-incidence of 'lock-out'. Even without that, however, the Cost, Availability,Quality aspect of health care is telling and has many chilling and deeply similar aspects to the NASA new project Faster,Better,Cheaper system.

That should be very, very worrying for those examining health care and trying to get high availability, high quality and low cost.

No system exhibits that so far. None.

Until we get something like the AutoDoc from Larry Niven's Known Space works, and we have a great trust in fully automated medicine all the way to major surgery, this will not change. It can, with very advanced technology and automated treatment systems, but those aren't past the 'wish we could do it' stage. That will change how we get medical care, by getting people out of the provisioning part of it... and that will have profound changes for society. Automation tends to do that to a system.

07 December 2007

Regression towards the mean and other fun things

In science one of the things that folks try to do in experiments is to figure out ways to reduce the impact of random variables. This sounds easy, but it is a very tricky endeavor as something that you may think of as fixed or being non-deterministic or static, may turn out to be an independent variable that will influence what you are measuring. The most famous of these is the gravity experiment which has an independent variable on Earth's surface called: atmosphere. When dropping something in an atmosphere there is a problem with mass to surface area ratios, that change findings apparently at random... soon you are trying to find out why two objects of the *same* mass don't fall at the same rate and looking at surface are and learning about dandelion seeds, maple seeds, and having to make things like lift and drag coefficients and account for friction... in no time at all you have forgotten about gravity and invented all sorts of wonderful ways to model the average air speed of a sparrow.

Those little extras prove very handy in understanding the world around you and soon it dawns upon you that yes! atmosphere is a problem. Ways to remove it as a variable include removing lift and drag out from the air via things like inclined planes and seeing if there is any way you can easily get all of the air out of an enclosure. You learn much about friction, air pressure and the tensile strength of materials.... and how to seal leaks. Friction, luckily, stays relatively the same for certain surface type interactions, can be categorized, measured and you can start measuring gravity! While mankind would, actually, also create the vacuum pump, good air seals and the pressurized hair spray, we would also put on demonstrations of things in a vacuum falling at the same rate. Of course stepping onto a surface without any atmosphere to worry about would take a bit:


The Apollo 15 Mission Commander David Scott would do a service to mankind and create a headache for conspiracy theorists everywhere. Of course evidence of the astronauts walking around demonstrated a lack of atmosphere: in walking around they would kick up lunar dust. It would fall back to the surface at a measurable speed that was pretty much beyond the SFX folks of the day.

Still, if you did *all* possible controlled experiments on Earth you would slowly shift to an average value of gravity. Initial values might be too *high* but that is the nature of first controlled experiments of a given type: you don't know what can happen and measurements can be off by quite a bit. And then you correct everything you can think of and the measurement drops far below expectations. This is an effect that takes place outside of science and is often known as the 'Rookie Year Effect' in sports. Given that each individual player in a given sport has their own subtle difference, when they become part of an established setting, they outperform: no one knows how to react to them until mid-season.

Rookies, having a great start, then do not vary their play and soon find themselves in a 'slump' as a sophomore. Part of the art of professional athletics of any sort is to vary your play and keep your opponents off-balance. In martial arts, this is known as 'not doing the same thing over and over': mix it up, don't do the same set moves all the time, understand your strengths and weaknesses and take advantage of openings. After a large number of professional matches, which your opponents have been studying, your ability to adapt in the ring/octagon comes to the forefront and you are now managing your own style. Professional team players do this, also, and those that post year-on-year solid performances are now performing to their mean as a professional player. One year measurements, given everything else being equal, are not true measurements of the mean or average ability demonstrated in a given milieu.

Or, at least that is the 'theory', which is nothing more than an observation of "star" rookies.

But just how true is this phenomena? In the media driven age we come to expect a lot out of highly drafted rookies and the spotlight of the media adds its own problems. So lets take a look at a few players to get an idea of how regression towards the mean shows up over careers. In baseball the slugging percentage is the sum total number of bases divided by the at-bats, with a single counting as 1 x bases, a double as 2 x bases, etc. and that at-bats as the number of times an individual is up as a hitter. If a player could get a home-run every time at-bat, they would have a perfect score of 4.000. Note that walks are excluded from the statistics.

Here is how that looks for a relatively well known player:

Source: Fan Graphs
Here the player has his scores and the average of the entire league as the blue line. Note the poor rookie start and yet the ability to 'recover' from that and to have a good slugging percentage over a decade that is above the average for the league. And you can, indeed, have an 'off-season' but for a prime player there is reversion to their form until career end, but even there some regression to the mean can be seen in a truly bad year then getting close to the normal range in the following year. The other thing to notice is that all of the MLB players are 'picking up their game' during this time period as the average is not flat and straight across, but slowly rising over time with a perceptible slope. Everyone is getting better over that time period.

A decade further on and we take another well known player's slugging percentage:

Source: Fan Graphs

Here the MLB slope is pronounced as compared to the previous graph, also that the percentage scale has been somewhat compressed to pull in the higher top number range. Here, again, the 'rookie jinx' of outperforming the opening year and then falling badly off the following is not seen for this statistic. This player started off very well and then saw a slow decline in slugging ability over his career, which was a long one. Also notice the two spikes that show up in the graph and that they demonstrate regression towards the mean in the following year.

Regression to the mean in sports usually indicates a random event or events causing either an unusual peak or valley in play, and it is rare that an individual can actually reach that 'higher level' of play.

Oh, the players? Reggie Jackson and Cal Ripken.

This phenomena is something that happens across fields and, as a concept, is as applicable to econmics, say, as it is to slugging percentages or the average size of seeds from siblings as compared to the parent stock if no other selection basis is used. Only if there are structural changes or selectivity against certain trends do we then see a changing in the mean, which then changes performance evaluation. As seen in baseball that slugging percentage has been on the rise for decades and, apparently, indicates a concentration on learning how to hit the ball and hit it so that bases can be gained.

Economies are similarly interesting as they represent an ongoing function of society and, thus, will reflect the trends of it. Take Haiti, which has suffered decades of structural problems and internal strife and conflict since the Nation was formed. While not having a long term statistical series, even a shorter one can see that this economy has had a period of stagnation that has lasted for some time:


Source: Indexmundi
That initial start is low, then heads upwards then down to the mean and there it sits for a few years before showing a couple of years of growth. For a number of economies the expectation is year-on-year growth, but Haiti must have another factor at play, something that retarded economic expansion until that factor changed.

A quick look at another trend gives us insight into how a deviation from the mean can be seen in another graph:

Source: Indexmundi
The rate of inflation in Haiti, already pretty high, had a definite spike over this timeframe, and that has an effect on the overall economy and its performance as seen in GDP. There are other factors at work, of course, such as trade, foreign relations and internal stability problems, but those tend to get reflected in statistics that measure discreet variables of the economy. When there is instability, prices tend to go up beyond expectation either due to reduced supplies or even basic unease about the future. When those factors moderate, inflation slows and the economy begins to expand again.

Part of my overall view of things is to take this sort of factoring into account. When there is stability in a system, it tends to have a mean (even if it is a derived one, like continued economic expansion at a given rate) that takes external factors to vary it greatly. Even then the regression to the mean is a powerful force that does not make itself known directly and is difficult to control in a running system. It takes a lot to change a system that is running well. For all the labor relations problems in MLB, the slugging percentage still tends to go up every year. Such an external factor influencing things can be seen in the following:

Source: Indexmundi
Very few economies on the planet can take an infrastructure loss of over $1 trillion, the cost of the lost real estate represented by the World Trade Center buildings that collapsed, and the insurance hit from those and the lost productivity and disruption of transit in a major city like NYC and still see a *positive* GDP growth for the year. In fact most of the loss in 2000-2001 was the collapse of the 'internet bubble' that had overpriced and overinflated technology sectors. That came winding down just as 9/11 happened to add in large scale infrastructure loss on top of purely economic loss from over investing in inflated stocks.

Yet the underlying strength of the economy shows up after those random factors had played out: economic growth returned to the mean and has been, steadfastly, bumping along that mean growth rate ever since. That is not what we hear in the media, which cites other factors like unemployment, the cost of a barrel of oil or pointing to bad investment in the form of $1 trillion in sub-prime loans in the private residential sector. From 2001 the lesson is clear: the US economy can take far in excess of a $1 trillion loss in a single year, and most likely near double that due to the crashing tech sector, and not even reach into negative economic growth. Put that over 2-5 years, which is the rate that those taking sub-prime loans will take to wash through the economy, and you get a near non-event with, at worse, a 1-2% drop in GDP growth over that period per year. Actually, if we have learned anything from 9/11, it is far better to take the hit all at once, than to spread it out: the ability of the economy to rebound to the mean is extremely powerful and putting a structural problem in over a number of years may have a deleterious effect in the longer haul. Do it all at once and handle it, and the economy bounces back; spread it out and you may cause worse problems for the long haul.

That may seem counter-intuitive, but the US has demonstrated this capability *and* needing to rev up for a war in Afghanistan at the same time, along with the tech bubble bursting and the huge loss of infrastructure not only from 9/11 but numerous companies going under. Sub-prime loans, restricted to poor loan products by lenders and taken up by unwise individuals to get more than they can afford will not see: a loss in jobs, two large holes in the ground in the center of NYC, a foreign war starting up, or an entire part of the economy's technical infrastructure going through a recession. Those people holding such loans will still have a source of income, absolutely awful lenders will get pushed to the side-lines and possibly out of business, and those wise institutions will reap the benefit of becoming known as solid lenders and reputable for not trying to entice individuals into debt they cannot afford. As very little industrial or commercial capacity will be hit by the sub-prime lending problem, the actual ability to grow while such a problem hits the economy is undisputed given recent US history when those very fundamental parts of the economy were the ones taking the direct attack internally and externally.

And then there are recovering economies, like that of Iraq:

Source: Indexmundi
Here structural changes have been an ongoing concern since the invasion of 2003, but the graph, itself, is representing something different as the mean coming into 2002 shows an economy of only $58 billion, which is in no way represented to the level it is rebounding to, which is nearly 60% higher. Even the 2006 fall-off is at a far higher level, than 2002, which looks very much like Reggie Jackson's career entry or Haiti's GDP in the 1999-2001 timeframe. While we can't tell much of Mr. Jackson's training and such in '68-70, we can say that it established a trend line near his mean for slugging. Haiti however, points to there being other influences on the Iraqi economy, so it is time to dig a bit deeper and see what else there is going on.

Here is the Iraqi GDP real growth rate:

Source: Indexmundi
This, if anything, is even more chaotic! The Iraqi economy *before* the 2003 invasion had experience a swing from +15% to -5.7% in a single year: over 20% change negatively between 2000-2001. The 2003 invasion brought a huge drop, again equivalent of the pre-war 2000-2001 drop, and then saw a change from -21.8% to +52.3% or over 74% change in the growth in the GDP in a positive direction, not only wiping out the drop over 2000-2003 (-44% cumulative), but then rebounding 30% above that. Mind you this is a Nation in disarray, getting its first taste of insurgency and yet we see a massive economic rebound beyond what any normal economy can sustain. Iraq fell and hard, losing that 30% gain and another 20% on top of that during the insurgency of 2004-06. Then after those wild swings the economy shifted to slightly positive.

If an insurgency is going to have a negative effect, it must sustain its ability to threaten and suppress the economy so as to inflict economic hardship on the population. The slew of infrastructure attacks from 2004-05 did that, but by 2006 the economy had begun to rebound from those losses. Obviously something else was going on, and that is told here:

Source: Indexmundi
When the coarse grained material leaves you at a quandary, head to the finer grained sources. Here the actual driver of the Iraqi economy, being its people, now comes forward as the source of change. On the per capita basis Iraq has seen a major jump in income even during the worst of the insurgency in 2004-06: apparently the concept of individuals being able to make a good living has hit Iraq and *not* in insurgency payments. The reflection of how well individuals are doing within the Nation gives lie to the the less than 10% refugee problem being a 'insoluble problem', as the people of Iraq have been making headway even as people were 'fleeing the insurgency'. When one puts down that premise, the reflection must be in the population and their ability to cope, or not, with the insurgency. It was by late 2006 that the idea of re-opening the factories there would start to take hold and by early 2007 that would begin, even before 'the surge'. Something more structural was happening in Iraq before 'the surge', apparently, that no one wanted to report on.

For all the insurgency, killing, loss of services, loss of infrastructure, hospitals closed, schools closed, bombings, murders, shootouts and the rest of it, there is one place where they have not been reflected and that is in these three key statistics which continue along their mean slope unchanged by all of that: death rate, life expectancy, infant mortality. In a Nation truly as bad off as it was painted these slopes not only are going in the wrong direction, but are *unchanged* throughout the entire affair from invasion through insurgency and all the way up to the start of this year. Apparently all that was going on had ZERO effect on the major dynamics of the Iraqi population.

This is one of the hardest lessons to learn about Counter-Insurgency work: if the people are not driven into despair in one of these three major areas due to violence, being deprived of basic services, or through some form of genocide you do not have much of an insurgency going. You soon go from glorified insurgent to a Royal Pain in the Ass that actually *can* be removed. An insurgency must cause a deviation from the mean that is long-term and disheartening enough to drive people to some form of radical solution. Usually that is how revolutions form: when government so ill meets the needs of the people of a Nation that the people just turn on the government and remove it as that is better than doing *nothing*. If you are very, very, very lucky and you have a generally well educated population used to civil culture, and that has not been under a reign of terror for a few decades, you will get a Philippine type 'People Power' movement. That is generally not the case in history as tyrannical regimes prefer populations to be on the uneducated side or so indoctrinated in the belief that they tyranny is 'good' that it will take a lot of abuse to dissuade them of that.

I have been saying for well over a year and going on two that Iraq is not going to hell because of the trendlines. These three, key items have been rock solid which points to an internal outlook of Iraq that is longer-term even into the Saddam era. For all the brutality, viciousness, corruption and just plain killing of Iraqis by Saddam, these trend lines did not change - the Iraqis were willing to put up with a lot so long as they got minimal services. The question was: would the breaking of the minimal services throw the Iraqi people into a revolutionary form of society attacking every-which-way and splintering the place? By SEP 2006 and the turning of Anbar on al Qaeda that answer was plain: NO.

By the end of 2006 each of these trend lines had shown evidence of shifting to a new mean: GDP Actual, Growth Rate and Per Capita. The underlying foundation of the Iraqi civil society that sustained the ongoing trend lines in death rate, life expectancy and infant mortality had withstood the shock of the insurgency and was showing signs of turning on it. 'The surge' was meant to drive out the worse external actors, clear out their internal supporters and start giving native Iraqi Army and Police room to entrench themselves within their society. The flow back of refugees is happening in those ethnically mixed neighborhoods because their neighbors *want them back*. There will be a residual that will not want to return: killers, supporters of AQI/JaM/Badr organization/criminal networks/Iranian Qods force, Ba'athists when not picked up by the former two categories, and those just wanting to start a new life somewhere else. Looking at those societal trendlines and their stability over time, my guess is that last will be a minority, but that is only a guess.

Now for a bit of graph analysis given equal time distribution! More fun than a body can handle this 'graph analysis' business...

Each of these graphs will be following an internal measured time that is equal along the Y-axis (with the numbers) and an equal spacing X-axis that I will keep unlabeled for now. These are all direct measurement graphs, not of derived figures, so it is something varying over time for each graph.


Here, between the interval of 900 to 2000 we see a recording with a mean line put in, so that one knows what the overall average is of the entire graph. Sort of like the Iraqi GDP per capita, isn't it? Especially from 1200 to 2000, you get something vaguely resembling it, although the recovery period doesn't outdo that 1200 period.

Perhaps a bit more like the US GDP Real Growth rate with a sudden drop off and recovery that never does get up to the previous growth rate? Maybe more like Reggie Jackson's 1972-'77 period, with that slump and then recovery, but not up to pre-slump slugging average. In any event you get the feeling that there is some regression to the mean, but with somewhat wide swings on either side of it, but never going outside of a limited range.

The next graph is also interesting in that regard:

A pretty good rendition of the 'rookie lead-in' ala Reggie Jackson, isn't it? And while there is previous record it also has a feel of Cal Ripken's career although not as many post-slump spikes nor the overall evenness of his career. More like Reggie's over the long-haul.

Some feel of the Haiti GDP or US GDP Real Growth Rate here, also, with a steep decline leading to a pretty flat recovery with minor ups and downs. There is a strong pull to that mean line through the graph, especially after that 'rookie' time entry.

This next one is more problematical, however, as the mean line obviously doesn't have any real impact on the measurement:
In fact you could draw the actual mean line between the fourth and fifth hash marks on the X-axis and still not get any feel for a regression towards or away from it. No feeling of stability at all here. Such a high amount of variability over time and the average is only that: an average with only a bit of draw power from entry to 2 and then some between 4 and where 5 would be. There is a feeling of a range that the measurement will not move beyond, but that is only like error bars on either side of an average reading. Pretty damned stochastic with preceding measurements not really indicating which way things will go, with sharp peaks and sudden declines; deep valleys and sudden rises. Might as well flip a coin after each measurement to see if you go up or down!

The next two represent two different measurements taken during time intervals as seen on the Y-axis going from 0 to 60, but each measures something different. Both are at the same place and the same time.
Absence or presence of a mean? There is a general slope as a mean here, obviously and the measurement trends along it. Here the Iraqi death rate, life expectancy, and infant mortality are the guides: a very strong mean. That said this graph has a wide deviation from it between time 20 and 35. If you flip the graph around and the insurgency could have done *that* to life expectancy between 2004 and 2006, we wouldn't be seeing much of a country left: just about everyone would be dead. I shudder to think what would happen if al Qaeda ever got their hands on nuclear devices, as I expect that life expectancy would see a major trough something along those lines.

Luckily this is not measuring that, so there is little problem there. But whatever it *is* measuring sure had a wide deviation from the mean slope and only recovered somewhat and then went down with the general slope again heading towards 0 on the X-axis.

Then there is the last one to consider:
Ah, such a lovely mean to show up! Right there, just above 10 is a mean that is almost rock steady and no matter how bad the trough is, it gets pulled right back up to the mean. Between approximately 28 and 33 you do have a very deep trough, but after that it not only gets pulled up again, but that 'rookie effect' shows up with a bit of over compensation. If these were baseball seasons and slugging averages, your career would have a few bad years, but then pop right back up to a rock-steady average.

And as these last two are co-measured, of different things at the same place at the same time, we should expect to see some correlation. There are some, but only on downward slopes where the top graph leads the bottom: around 12-15 and 20-30. But it looks like the bottom one leads the top in a few areas: 15-18, 33-34, and that spike around 36. So there are weak correlations, but nothing that I would pin as definitive, with sometimes the first leading the second and just as often the second the first.

Well, there you go having some fun with regression towards the mean... what?

What are those last series of graphs? I originally featured those in this post, and did a bit of photoshop work to remove original axis labels and such so that a feel could be had on just how graph reading and interpretation could help distinguish between strong and weak correlations on data. The last two I decompiled from a larger graph where they were put together, and I pulled them apart to better examine them... most folks have a problem doing that, so removing all sorts of extraneous information allows the data to show clearly through. Still, it was down and dirty photoshop work.

So I will now put them up so you don't have to go wandering back to that previous post:

Courtesy of our friends at NASA.

Again from the folks at NASA.


Courtesy of Geocraft.com.

Yes, that last one is the compiled graph with the 0-8 first one being in 1,000 of ppm of carbon dioxide in the atmosphere and the lower one being mean global temperature by adding 12 degrees Celsius to the X-axis.

Notice the examined results leading to the conclusion that there is, at best, a weak correlation between global temps and carbon dioxide?

The graphs do not lie.

Apparently those in the global warming religion do. Or they don't know how to read a graph...