Black Swan events: Sadly, global IT projects are turning the rare into the everyday

by Michael Arbow, MBA

Partner, RSD Solutions Inc.

www.RSDsolutions.com

info@RSDsolutions.com 

 

In 2007 Nassiam Nicholas Taleb had his book “The Black Swan – The Impact of the highly improbable” published and with it brought to the fore the term “black swan event”.  As defined by Mr. Taleb it is an event that is either never considered or considered extremely unlikely to occur, but when it does occur it is not surprising and usually easily comprehendible (we should have seen it coming).  The book goes onto explain many examples of black swan events such as 9/11 and Microsoft and events which the mass media feel are black swan events but are not – typically stock market gyrations.  This later use of the term is now being applied to large-scale global IT projects (in excess of $170 million USD) where “it found that while most projects ran less than 30% over budget, a sixth ended up costing on average three times (!!!) as much.  The study also raised concerns about the adequacy of traditional risk-modeling systems to cope with IT projects, with large-scale computer spending found to be 20 times more likely to spiral out of control than expected."  Clearly these over-runs are not a black swan event in the more pure definition.

 

This situation beckons a lot of questions surrounding risk modeling, the varied expertise that forecast costs and management seeing an anomaly and not recognizing it as the trend (fact?).  As the world settles into a new century with new paradigms and new risks, does your firm still consider (hope) that the rare is still rare and not the signs of a trend? 

 

For more on IT’s Black Swans check out this BBC news story:  http://tinyurl.com/3ruj4yf 

For more on Mr. Taleb’s excellent seminal work follow the link:  http://tinyurl.com/3h64jdl

 

Models - Part 4

by Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.RSDsolutions.com

info@RSDsolutions.com 

 

In the initial blog in this four part series about models, I talked about how models were a balancing act.  In the first blog I discussed how models were a balancing act between being too general (and useless), and too specific (and also useless).  In the second blog I inferred that models were a balancing act between hiding things and not hiding things.  In the third blog I implied there needed to be a balance between information provided by models and the actions taken by managers.  (OK, that is a bit of a stretch, but I think you get my point.)

 

In this last blog for the series I would like to bring up the balance between keeping a model static and adapting it to changing conditions.  A model that is constantly changing (through time) will not provide much guidance or comfort, as it will not be capable of being tested under different conditions.  Likewise a model that never changes to account for developments in knowledge or industry practice will also not be of much use. 

 

The upshot of all of this talk about models and balance is that using models is really tough!  Too often regulators, managers, and other stakeholders believe that if the right model is selected that all will be great.  That is far from the truth and the reason why we need to get back to an emphasis on managing and less of an emphasis on modeling. 

 

Models - Part 3

by Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.RSDsolutions.com

info@RSDsolutions.com 

 

What exactly do models do?  They tell us about part of a system (acknowledging – since you did read my previous blog – that they hide stuff).  But what do models actually do?  What actions do they perform?

 

Of course models do not act.  Hopefully however they cause someone to act, since risk management is not a spectator sport.

 

Do you act based upon your models, or does your risk management department hope the models themselves take care of the issues and seize the opportunities by themselves?  Which is it?  Is risk modeling the be all and end all of risk management?  Is there a disconnect between the people who build the models and the people who need to act based upon the models? 

 

Wow – if only models could actually do more.  

 

Models - Part 2

by Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.RSDsolutions.com

info@RSDsolutions.com 

 

Sunny Marche is a colleague of mine at Dalhousie, as well as one of the Associates at RSD Solutions.  Dr. Marche is an incredibly thoughtful person, and combined with his interest in knowledge development and his breath of academic interests, it makes for a powerful intellect.

 

Sunny was explaining some of his work on models in database management, when he said something that embarrassingly I had not considered before.  Namely, he said that models are devices for “hiding things”.  In other words, models leave out details that are not believed to be germane to the problem at hand.  In reading this here, it of course seems obvious, but does the implication?

 

It is generally acknowledged that we live in a complex world.  (I will avoid the easy reference to my upcoming book on complexity theory).  A complex world by definition (and by fact) means that you cannot separate the parts from the whole.  In a complex world you need “systems thinking”.  Studying the parts does not help you find solutions, and indeed is probably counterproductive.

 

In that context, thinking about models as devices “that leave parts out” seems to be a stupid thing to do.  The implications of leaving things out are that you are ignoring the systems characteristic of all aspects of business.

 

What do the models that you use leave out?  Is risk management immune from systems thinking, and thus it is ok to leave stuff out?  I think not.

Models - Part 1

Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.RSDsolutions.com

info@RSDsolutions.com

  

On a warm summer afternoon of a long weekend, models for some strange reason popped into my head.  I am in the midst of doing researching for my book on complexity, and thus it may be natural that models, and “concepts of models” are swimming around that thing that is on my shoulders.  A lot has been written about models (in fact I have written a lot about models – both in their praise and in their condemnation), but I think it is always healthy to resurface a few ideas that have not been thought about in a while.  Thus this four part blog series on models.  Unfortunately I understand a very high proportion of the science jokes on “The Big Bang Theory” and thus I am of course talking about mathematical models and not runway models – which of course would be a much more interesting topic, but not all that germane to risk management.

 

Richard Levins, the mathematical ecologist (according to Wikipedia) is claimed to have said “ models tend to be so general that they cannot make predictions about particular systems, or so detailed that they merely rephrase what is already known about a system.”  (This quote is from Ants at Work: How an Insect Society us Organized, by Deborah Gordon, Free Press, 1999)

 

I believe that the same could be said about our risk models, and finance models in general.  They are so vague to be useless, or so detailed that they simply replicate the past (and are subsequently useless for the current or future contexts).

 

Models should be a balancing act, and one dimension of that balancing should be between the general and the specific.  Where do the models that you use lie on this spectrum?  Or were they models that were handed down from a regulator or a textbook, or even worse, a committee?

VAR Trend is Your Friend (or at least a good early indicator)

by Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.rsdsolutions.com

info@rsdsolutions.com 

 

For some reasons a lot of stuff has come across my desk recently that is bashing Value at Risk (VAR) and saying that it is a symbol for all that is bad in risk modeling.  I think it is time for a time-out.  As risk specialists, we also have to be optimists and look for the good in everything. 

One thing that I think that VAR is good at is indicating change.  I too recognize the limitations of VAR.  (One of the reasons so much is crossing my desk is that I teach several different courses in financial risk and also in Enterprise Risk Management.)  The limitations have been well documented. 

What is less well recognized is that VAR is still a very useful tool.  One of the ways to use VAR that helps to maximize its usefulness while minimize its weaknesses is to follow the trend of how the company (or project) VAR is tracking.  Looking at the trend, rather than the absolute number, can help you to ascertain when a shift in risk exposure has occurred.  It will also help to base-line out the traditional faults such as the use of parametric statistics or the use of the incorrect distribution. 

Tracking the trend of VAR is not a perfect remedy.  Watching the trend of a bad and inaccurate model, will still give useless and potentially misleading results.  However if a decent VAR model is constructed, and if more attention is paid to the trend than the absolute VAR number, then more useful (and accurate) information will be generated.

The Judgment Deficit

by Rick Nason, PhD, CFA

Partner, RSD Solutions Inc.

www.rsdsolutions.com

info@rsdsolutions.com 

 

On my most recent plane trip I read a Harvard Business Review article by Amar Bhide that I have been meaning to read for some time.  The article is titled “The Judgment Deficit” (HBR, September 2010, pp 44 - 53).  

In this article, Bhide essentially points out that we have surrendered judgment to models and rule based thinking.  While he points out that there are times and situations where rules or models are appropriate, they can never capture the complexity and value of human judgment.  I wholeheartedly agree with him.  The article points out deficiencies in having computer models make lending decisions as well as other examples from the banking sector, but the surrender of human judgment to models (and even worse -  audits) is endemic throughout risk management – whether in the financial sector or not. 

As Bhide argues, it is time to get back to basics and get back to using human judgment.  A great judgment to start with is to read Amar Bhide’s article.

Pentagon Political Risk Model: $125 million. Reliability: Ummm,...

by Michael Arbow MBA

Partner, RSD Solutions Inc.

www.rsdsolutions.com

info@rsdsolutions.com

 

Over the past 4 decades the US Pentagon (Department of Defense) has spent untold millions attempting to build a predictive political risk model, the latest iteration of which cost around $125 million USD.  Despite the vast amounts of data gathered from world political experts and the complexity of the algorithms, they failed to predict the recent events in Egypt.  Luckily the direct impact on the US in this particular instance is minimal; for now. 

Rick Nason, a partner at RSD, has pointed out that risk can be simple, complicated and complex.  When people and human emotions are involved it seems that complex risk management wins the day.  Is your organization operating under a “tick the box” risk management system?  Alternatively, has your organization put faith in algorithms? 

For a link to “Pentagon’s Prediction Software Didn’t Spot Egypt Unrest” from Wired Magazine click:

http://tinyurl.com/6jc3zzc