Big banks are leaving the building when it comes to the mortgage market: In 2007 commercial banks made 74 percent of all mortgages versus half today. Big investors and hedge funds have largely left the building when it comes to investing in residential real estate.