When the leaders of Irvine, Calif.-based wealth management firm Keller Financial Group decided to charter a bank three years ago, they could not have known what gut-wrenching times lay just ahead.

“In 2007, when we opened our doors, there was this pause and we said, ‘What did we get ourselves in to?’ ” said Scott F. Kavanaugh, the chief executive of the bank.

First Foundation Inc., which opened for business Oct. 1, 2007, did indeed go through a crucible. But it has emerged in good shape—thanks largely to its roots as an advisory firm, said Kavanaugh.

The holding company contains First Foundation Bank and the advisory business, now called First Foundation Advisors. Assets under management, which stood at $1.2 billion around the time First Foundation’s opening, plunged to around $900 million during 2008, in the teeth of the market crash; but they have rebounded to $1.4 billion.

Total assets at the bank, meanwhile, have posted steady, quarter-to-quarter climbs. From $36.6 million at the end of 2007, they had risen to $238 by the end of 2009. The net loans and leases it reported to the FDIC jumped to $197.9 million at the end of last year from $89.2 million 12 months earlier.

As of end of March, said Kavanaugh, the bank had $226 million in loans and a similar amount in deposits.

“In all honesty, it’s been better than we ever thought,” he said.

What’s more, the company has signed lease in Pasadena, for a second branch, which is to open in August. It will allow First Foundation’s expansion into the greater Los Angeles market, Kavanaugh said.

Investment advisory clients have clearly buoyed the bank: As much as 70% of the deposit base has come “as a result of referrals from the advisory side,” he said.

First Foundation is not technically a private bank. But its trust business and the fact that it was spawned by an advisory business that caters to high-net-worth clients might make it seem that way, Kavanaugh acknowledges.

“Our model is what a U.S. Trust used to look like,” he said, referring to the company that focuses on very wealthy clients, and was bought in 2000 by Charles Schwab Corp., only to be sold in 2007 to Bank of America Corp. [BAC]

The market and economic environment may have dealt First Foundation a bad opening hand, but they also created the opportunity to attract clients and talent from bigger organizations, according to Kavanaugh.

Its advisory business has hired executives from larger organizations. Overall, employment at First Foundation has risen above 70, and will staff up more before the Pasadena office opens.

First Foundation made a splash when it was created: Although wealth management firms are often targeted for acquisition by banks, advisory firms don’t often overturn the formula by opening banks.

Keller Financial Group’s leadership had received inquiries from several large banks, but opted to control their company’s destiny instead. The group had already worked with Kavanaugh as a result of their investment in Commercial Capital Bancorp Inc., of Irvine. Kavanaugh helped to found and run that bank, which was sold in 2006 to Washington Mutual Inc.

Kavanaugh has cited what he sees as an opportunity to gain investment clients who are dissatisfied with wirehouses and other big firms and want more personal service. Since the beginning of 2008, First Foundation’s client list has risen more than 10%, to about 1,100, according to Kavanaugh.

But now that big institutions are stabilizing after mergers and reorganizations, it will likely be harder to win clients away from them, according to Alois Pirker, a senior analyst at the research firm Aite Group LLC.

“There has been a window of opportunity, but it is closing as large firms move through their integration efforts and bring their ships to an even keel,” he said.

But it may still be realistic to recruit wirehouse advisers, Pirker said. A recent Aite survey showed that a mere 15% of advisers within wirehouses are not contemplating leaving. And 20% of the respondents said there is more than a 50% likelihood that they will break away.

“So the appetite is still there,” said Pirker. “The messaging you hear from the large firms is that the attrition is over, but that may be a temporary state.”