One afternoon this spring, a dozen or so employees lined up in front of Freedom Mortgage’s office in Mount Laurel, N.J., to get their picture taken. Clutching helium balloons shaped like dollar signs, they were being honored for the number of mortgages they had sold.
Freedom is nowhere near the size of behemoths like Citigroup Inc. or Bank of America Corp.; yet last year it originated more mortgages than either of them, some $51.1 billion, according to industry research group Inside Mortgage Finance. It is now the 11th-largest mortgage lender in the U.S., up from No. 78 in 2012.
Its rise points to a bigger shift in the home-lending business to specialized mortgage lenders that fall outside the banking sector. Such nonbanks, critically wounded in the housing crisis, have re-emerged to become the market’s dominant players, with 52% of U.S. mortgage originations, up from 9% in 2009. Six of the 10 biggest U.S. mortgage lenders today are nonbanks, according to the research group.
They symbolize both the healthy reinvention of a mortgage market brought to its knees a decade ago—and how the growth in that market almost exclusively has been in its less-regulated corner.
Since the crisis, banks have pulled away from mass-market mortgages to focus on wealthier consumers. Today, nonbanks like Freedom often represent the only route for first-time buyers and moderate-income families to get a mortgage.
Postcrisis regulations curb bank and nonbank lenders alike from making the “liar loans” that wiped out many lenders and forced a wave of foreclosures during the crisis. What worries some industry participants is that little has changed about nonbank lenders’ structure.
Their capital levels aren’t as heavily regulated as banks, and they don’t have deposits or other substantive business lines. Instead they usually take short-term loans from banks to fund their lending. If the housing market sours, banks could cut off their funding—which doomed some nonbanks in the last crisis. In that scenario, first-time buyers or borrowers with little savings would be the first to get locked out of the mortgage market.
“As long as the good times roll on, it’s fine,” said Ed Pinto, co-director of the Center on Housing Markets and Finance at the American Enterprise Institute. “But all I can say is, we’re in a boom, and you cannot keep going up like this forever.”
Already, cracks are starting to show. Despite good economic news, home sales are slowing because of sky-high prices, and refinancings are drying up as interest rates rise. Many of the biggest nonbanks today didn’t exist a decade ago, which makes some industry participants wonder how well they would navigate a new crisis.
Freedom was just a small lender in the last crisis. When it became hard to borrow money, Freedom Chief Executive Stan Middleman embraced government-backed loans on the theory they would offer more stability.
As Quicken Loans Inc., the biggest and best-known nonbank, grew with the help of flashy technology and advertising campaigns, Freedom stayed under the radar, buying smaller lenders and scooping up other companies’ huge portfolios of loans, often made to relatively risky borrowers.
Mr. Middleman is fond of saying that one man’s trash is another man’s treasure. “I always believed that, if somebody is applying for a loan, we should try to make it for them,” said Mr. Middleman, in shirt sleeves and with his company-ID on a lanyard around his neck, in an interview at his office.
In the first six months of this year, Freedom was the No. 5 lender of Federal Housing Administration mortgages and the No. 6 lender of Department of Veterans Affairs loans, according to Inside Mortgage Finance.
FHA loans have traditionally been made to first-time homeowners with weaker credit histories, and they tend to have higher delinquency rates. VA loans, for military veterans, can offer perks such as no down payments.
But if a borrower stops paying, there is usually a lag until the government guarantee is paid. In the meantime, the mortgage company servicing the loan has to pay investors each month as it waits for government reimbursement. In that scenario, some nonbanks could quickly run out of cash.
As for Freedom, Mr. Middleman said its capital levels are strong: “We’re better capitalized than a lot of banks.”
Last year, nearly three-quarters of Freedom’s loans were FHA or VA, according to ComplianceTech’s LendingPatterns.com. Across the industry, about one-quarter of loans were FHA or VA.
Mr. Middleman, 64 years old, started Freedom Mortgage in 1990, in a shared office behind a Department of Motor Vehicles branch in Cherry Hill N.J. While some other large nonbank lenders are backed by private-equity money, Mr. Middleman and related trusts own Freedom, estimated to be worth between $1 billion and $1.5 billion, according to people familiar with it.
The atmosphere at the company’s headquarters is at once casual and high pressure. Workers decorate their desks with stuffed animals representing their team names and call themselves “American dream makers.” Posters praise those who make the most sales. Some young workers get taken on laser-tag and bowling outings.
Some former workers said they found the culture off-putting. Turnover is high, and loan officers in branches said they competed against Freedom’s own call-center employees for the same customers.
Mr. Middleman touted Freedom’s work culture: “We want it to be fun when people come to work.” He said competition, even among co-workers, “sounds like the American way.”
Ginnie Mae in June curbed the ability of Freedom and two other lenders to securitize certain VA mortgages. The agency had raised concerns that Freedom was repeatedly refinancing the loans, which can result in veterans paying unnecessary fees and higher interest rates.
Mr. Middleman said Freedom makes loans only if they benefit borrowers and that refinancings are driven by consumer demand.
Chris Houvouras, who lives in Wilmington, N.C., is grateful that Freedom issued his mortgage loan three years ago, even though he had just accidentally missed a student-loan payment. “They helped me out of a jam,” he said. He has declined offers in frequent calls from Freedom associates of a refinance, because his broker said he’d pay the same amount in the end after adding in fees.
“They weren’t rude,” Mr. Houvouras said. “But it was like somebody wanted to check a box that they had gotten somebody to refinance.”
During Freedom’s rapid growth, state regulators have reprimanded the company for administrative slip-ups such as allegedly failing to file quarterly reports on time. A number of customers for whom Freedom handles property-tax payment have complained to the Consumer Financial Protection Bureau that Freedom has sometimes failed to pay their property taxes on time, a mistake that can lead to foreclosure.
Mr. Middleman acknowledged that paperwork can sometimes fall through the cracks, especially when loans are transferred between lenders. “I never thought that we were perfect, but I know we can get better every day,” he said.
While consumer advocates have voiced criticism of Freedom, some also say the company plays an important role in offering loans to minorities and moderate-income borrowers.
“They work directly with the clients to make sure that what they’re offering matches the clients’ needs,” said Andrea Haughton, director of homeownership at the nonprofit Community Housing Innovations, “while a lot of mortgage lenders just offer a product.”
Freedom could be due for another reinvention. Mr. Middleman said he believes an improving economy will diversify its pool of borrowers; the lender’s proportion of FHA and VA loans has shrunk this year.
He recently tapped a son, Mike, to hone Freedom’s strategy to connect with millennials, but demurs on questions about his own future at the company.
“I’m young,” he says, “and going to live forever.”