Mortgage Bank Uses Tech Tools to Support Growth

By Ricky Caldwell

Founded in 1988, Cornerstone Home Lending is a privately owned mortgage bank that funds more than $5 billion in annual loans. In-house processing, underwriting and funding enable Cornerstone’s mortgage advisors to deliver competitive, streamlined financing and on-time closings with best-in-class customer service. The company currently has 122 branch locations in 22 states and almost 1,500 employees.

To support our ambitions to grow to $10 billion in annual loans over the next few years, we needed a solid technology platform. Despite virtualizing previously, we had developed a large footprint of physical servers that required substantial resources and investments to manage and maintain.

We wanted to maximize use of our Microsoft enterprise license agreement, so we began implementing Hyper-V, as we thought it could be a good fit for us in the virtualization arena. We brought in an outside vendor to implement the solution, but it never panned out the way we expected. The virtual machines were bouncing around from one host to the other. The Failover Cluster Manager in Hyper-V could never find a home it was happy with, and it was constantly moving around.

It soon became a nightmare. We had moved several of our production servers into this environment, and it was completely unstable. The vendor kept throwing hotfix after hotfix at us until we threw up our hands. We then moved everything back to either a standalone Hyper-V host without a cluster or re-enabled the machines that we had running in our old VMware environment.

Due to this poor experience, we launched a thorough investigation into competitive virtualization offerings that would provide a stable and reliable architecture. In the end, we chose VMware.

Picking a hypervisor was an important step in the right direction. Once we made the decision to move platforms and vendors, the question became: “How do we not end up here again?”

As we evaluated hypervisors, we also evaluated different tools to manage our deployment. We wanted the ability to forecast storage resources, watch compute resources, run analyses and show the business the budget for the next one-to-three years based on the rate of experienced—not assumed—growth.

vSphere with Operations Management fit all those needs and was easy for us to use right out of the box. This allowed us to act proactively and give the business a clear model of projected costs, along with a timeline for change.

From what some refer to a “single pane of glass,” we are able to look into the health of an entire environment. We get alerts about LUNs (logical unit numbers) over activity, high I/O, processor spikes and memory leaks. We can view historical data to see if there is a problem server, application or network issue.

Some in-house written applications have, over time, shown a propensity to have runaway SQL processes. Luckily, before close, in the second month of implementation, we saw the database spike processor and memory utilization. Because we had historical data, we could pinpoint when this was occurring during the batch processing jobs on the server. The development team was able to identify a bad section of code based on what we were able to give them. Overall, that insight saved us many hours of downtime or server reboots by looking at historical performance data.

The ability to use vSphere with Operations Management on a daily basis makes the job of an operations staff member easier. The team can focus on the health of the server farms and clusters, while proactively monitoring the month-end close for any issues that might arise.

We couldn’t do our job to the best of our abilities without the tools we have in place. We value the relationship with VMware and the other partners that enable us to become a consistently growing full-service lender.

Ricky Caldwell is the director, Server Infrastructure and Architecture, at Cornerstone Home Lending.