Neil Barofsky, a former special inspector general overseeing Troubled Assets Relief Program aimed at strengthening financial institutions during the financial crisis, calls the program a failure.

In a talk on his new book, “Bailout: An Inside Account of How Wall Street Abandoned Main Street While Rescuing Wall Street,” in New York on Thursday, Barofsky emphasized how TARP’s $700 billion put the balance sheets of large financial institutions first and did not live up to its promises to help up to 4 million homeowners.

While President Barack Obama initiated a $50 billion program to help those homeowners, only about 830,000 of the 4 million homeowners targeted have seen any relief, Barofsky said. The program has also “slowed to a trickle,” Barofsky said, at a rate of just about 6,000 net additional program modifications per month.

“It would take another 43 years at this rate to fulfill that original goal, and just a fraction of the $3 [billion] or $4 billon has actually been spent,” Barofsky said. “That’s about the same amount of TARP money that went to American Express as went to all the homeowners, that were part of the reason this bill got passed in the first place.”

Barofsky admits he was reluctant at first to take the position as special inspector general tasked with overseeing TARP’s deployment in the weeks immediately following the financial crisis. But his boss at the time, Mike Garcia, the U.S. attorney for the Southern District of New York, convinced him that it was what he had to do.

Garcia reminded Barofsky of the prosecutors in their office who had also stepped up after 9/11 to leverage their experience investigating terrorism. Barofsky, then a federal prosecutor who had just launched the Mortgage Fraud Group, had the right experience for the position, he remembers Garcia telling him, and he argued it was his patriotic duty.

“He said, ‘This is the economic equivalent of 9/11, and you have to take this position,’” Barofsky said.

Barofsky started in his new position on December 15, 2008, following a nomination from President George W. Bush, confirmations by the Senate and a swearing in ceremony in Treasury Secretary Hank Paulson’s office.

The approximately two years Barofsky spent in that position held some surprises, he said. The oversight body that Barofsky was appointed to, the Special Inspector General for TARP, known as SIGTARP, was tasked with a two-part mission: law enforcement to prevent fraudulent use of the government funds, and overseeing the Treasury and reporting to Congress on the TARP program’s progress.

And while Barofsky anticipated working mostly to prevent fraud, he said he quickly found that most of his time was spent working to get more clarity on what was happening with the TARP funds. Barofsky’s efforts were met with hostility, which included on expletive-filled conversation with Treasury Secretary Timothy Geithner, Barofsky recalls in his book.

“When I started pointing out areas of conflicts of interest of areas of vulnerability to fraud, I was given figuratively the pat on the head and, in an incredibly condescending tone, saying, ‘You just don’t understand. You’re a prosecutor,’” Barofsky said. “’You don’t understand that these are banks and that they would never ever risk their reputations,’ which were the precise words that were used, ‘by putting profits over the public purpose of these programs.’”