What's the common thread that connects the subprime crisis, stock prices and the price of oil? One answer—the standard macroeconomic one—deals with how asset prices and the real economy are linked. But there is another, more subtle answer. Each of these current market issues is an illustration of important biases that influence investor psychology. And each holds lessons for how we, as advisors and clients, should think about long-term investment planning.

Researchers continue to uncover more evidence that our brains are hard-wired to react to risk in specific ways that might have worked well in our evolutionary past, but that are not necessarily adapted to modern financial decision-making. And as we consider the subprime mortgage crisis, stock prices and the price of a barrel of oil, two behavioral biases come to mind: overconfidence and loss aversion. These biases are so powerful that they seem to act as a metaphor for the behavior of major markets in recent years.

Start with overconfidence, which Daniel Kahneman, the Nobel Prize-winner for economics, has described as a tendency to construct forecasts that are "too rosy." The evidence for overconfidence in human decision-making is quite strong and extends well beyond the domain of investing. For instance, most American drivers consider themselves to be above-average. So do most students. Doctors, lawyers and other professionals are also prone to overestimate their own skills. The same is true of managers and executives. How can some chief executive officers have top-notch financial and legal talent and still often fail to create shareholder value with mergers and acquisitions? Overconfidence may be the problem.

It's easy to see how overconfidence pervades the stock market. Money managers and advisors are paid for their expertise, and most promote—and believe in—their superior skills. But only half of them consistently perform above average peer benchmarks. Among investors, overconfidence plays out in other ways, such as chasing short-term performance and hot asset classes. "It's different this time" is the mantra of the overconfident investor. The late 1990s "TMT" bubble—the surge in technology, media and telecommunications stocks—is a classic example of market psychology driven by overconfident forecasts. The subprime crisis was in part due to problems of outright fraud and market manipulation. But the greater driving force was an all-too-human skill at creating overly optimistic forecasts. Real estate agents promoted the idea of ever-rising home prices, even though the long term evidence in favor of residential housing as an asset class has never been particularly strong. Homeowners bought into the notion that they could live off ever-rising home equity through borrowing. A new art form—flipping—emerged among real estate investors. Mortgage originators took too optimistic a view of homeowners with weak credit. And Wall Street underwriters and credit-rating agencies did the same in estimating default rates and interpreting underwriting standards. In the subprime crisis, it was hard to find anyone who wasn't overly optimistic about the future.

Turn now to the latest example of market exuberance—oil and other commodities. From the optimist's perspective, this time it is truly different. The reasons include growing demand from China, India and other developing economies, leading to perhaps permanently higher commodities prices. There is also the "peak oil" argument that the global rate of extraction has either reached its maximum or soon will.

Investors, both retail and institutional, are flooding into the marketplace for oil and other commodities. The ostensible reason is that commodities offer portfolio diversification, which is true. The difficulty today is distinguishing between those investors with a strategic view of these assets and those whose strategic views are really just a short-term forecast that's "too rosy" in its expectation of double-digit returns.

If market participants overreact on the optimistic side in good times, it seems that the overreaction is even more profound in periods of falling stock prices. Researchers have noted a psychological tendency toward loss aversion—a tendency to overweight losses relative to gains. In psychological terms, a $1 loss is about two-and-a-half times more painful than a $1 gain is pleasurable. From a psychological perspective, a gain of $2.50 or so is needed to offset a $1 loss on a portfolio. Loss aversion appears to be at the root of many of the worst types of investment behavior: selling out of the market entirely; abandoning asset classes based on short-term returns; focusing on specific losing investments rather than on overall portfolio performance. If you thought real estate investment trusts (REITs) were a strategic asset class when they earned 30% a year, and then wondered about their long-term merit when prices were falling, you too have ridden the overconfidence/loss-aversion roller coaster.