As the calendar turns the page to another year, 2013 offers the four largest U.S. wealth management businesses the opportunity to recapture some of the assets and talent they have been ceding to other business models including regional, RIA and independent firms.

But those wirehouse firms—including Merrill Lynch, Morgan Stanley, UBS and Wells Fargo—need to pay close attention to two areas in order to accomplish that: profitability and financial advisor compensation.

“Ultimately, what the market is really expecting from the wirehouses is returning to growth and real growth,” Alois Pirker, research director at Boston-based research and consulting firm Aite Group, said. “So far what they’ve done is buying themselves to growth, handing out sign on bonuses. In the long run, that’s not where you really want to be in order to be profitable.”

The need for the large firms to increase their hold on the market comes as studies increasingly show these firms losing their grasp. Just one-third of advisor moves now are from wirehouse to wirehouse, according to Aite Group, with about one fourth of all of financial advisors eyeing a move toward independence. And even more startling is the survey prediction by industry research group Cerulli Associates that wirehouses will lose 7% market share in the next three years.

The wirehouses have already begun their efforts to climb back, Pirker noted, particularly with the compensation plans they have unveiled for 2013 that call for more profitability from advisors. But each firm also must contend with certain challenges. Morgan Stanley, for one, has to find its footing after focusing on its merger with Smith Barney for the past two years.

“It’s not clear yet what the post merger profitability will be,” Pirker said of the firm, which made cuts to its regional and management structure this year while continuing to integrate technology platforms. “I think they’re certainly working toward the idea that they will be more profitable in the end once this whole thing is done.”

Merrill Lynch, meanwhile, has a track record of profitability, Pirker noted, but has come to rely on sign on bonuses and retention packages to keep their top talent in place. “For that, you need to be able to rely on your top producers to stay and for nobody to poach them from you,” Pirker said. 

And as UBS is devoting up to 10% of its net income for special advisor compensation, according to Pirker, and Wells Fargo now serves as the only wirehouse where advisors can also operate independently, the competition among the wirehouses to differentiate their models remains strong. But it is also the other regional, RIA and boutique firms that are also getting more interest from advisors searching for new homes for their businesses. 

“If you are a big producer and have a big book, you certainly have more options open,” Pirker said. “I think the advisors have become a lot more savvy about choices and a lot more educated.”

Recruiter Danny Sarch, president of Leitner Sarch Consultants in White Plains, N.Y., said that while he is agnostic about where his clients should go, more and more ask about firms offering their own spin on the independent models including HighTower, Dynasty Financial Partners or Focus Financial Partners “to see what they’re missing.”

But the wirehouses continue to have a hold in the industry. Even with the LIBOR issues, UBS had the strongest year of the four wirehouse firms in 2012, according to Sarch.

Recruiter Bill Willis, president and CEO of Willis Consulting in Palos Verdes Estates, Calif., said he also sees Morgan Stanley facing the most challenges as it works to boost post-merger advisor satisfaction in the coming year. Wells Fargo, meanwhile, is quietly in a strategic position, he said, to grow all sides of its brokerage business that also include the bank and independent sides. “They can be most active where it pays to be most active,” Willis said.

Ultimately beyond 2013, the long-term competition among the wealth management firms will not be settled by recruiting success, but by how well they can solve the key dilemma facing the industry now—replacing an aging advisory force. 

“To me, the biggest question in the industry is the aging of the best producers,” Sarch said. “And the firm that is best able to address that in terms of succession planning and in terms of training will be the winner.”