Unlocking Access to CustomerBy Mel Duvall | Posted 2006-02-07 Email Print
The firm reinvigorated 420 financial programs stored on mainframe computers. How? By buildingfrom scratchWeb services that can handle millions of interactions a day.-Service Functions">
The actual case for developing Web services was a no-brainer. As reliable and robust as Merrill's mainframe infrastructure was, it was also fraught with limitations. As new Internet-based applications, such as self-help credit card balance checks, were being developed, programmers needed to access data locked in Merrill's mainframe vaults. It was difficult to tap directly into those vaults using non-mainframe-based software.
In fact, it's estimated that 90% of all new development costs involve the plumbing to integrate mainframe applications with newer platforms. Merrill had instead taken the approach of copying that data into Oracle, Sybase or Microsoft SQL databases, which could more easily integrate with server-based applications.
But the copying of the data created two problems: Copying tends to be unreliable because of disk errors, read errors and running-out-of-space errors. Second, the data could be out-of-date as soon as it is copied.
For example, a client making several trades would have to wait until the following day to see an accurate balance in his account. As a result, a client might make a trade believing there were adequate funds, but that trade would have to be rejected because, in fact, the funds were not available.
"A lot of money was being spent copying data to Oracle or SQL servers, and the frustrating thing was this information was available and up-to-date on the mainframeit was just difficult to get at," Crew says.
Web services seemed the ideal solution to the challenge. Through a set of widely accepted standards, such as XML, which uses tags to describe data and which can in turn be understood by any XML-aware application; and SOAP, which provides a means for a program in one kind of operating system to communicate with a program in another kind of operating system, it was possible to open up Merrill's mainframe vault to modern applications.
In a simple scenario, a client using an online application to look up his credit card balance submits a request via a Web browser. The mainframe runs the requested operation (obtain credit card balance) and sends the information back to the client via SOAP. XML makes it easier for the mainframe to interpret the online application, and in turn assists in allowing the online program to understand the response delivered by the mainframe.
In 2000, Crew formed a team within the database group to begin exploring the relatively new CICS Transport Control Protocol/Internet Protocol (TCP/IP) capabilities. As a result, the group wrote a tool, using CICS TCP/IP, to replace an existing mainframe tool used for getting real-time market data to the mainframe.
The existing tool employed the more traditional Systems Network Architecture (SNA) protocols, a set of network protocols developed by IBM, and was used to allow a Merrill Lynch customer accessing his account online to get pricing of a traded security directly from the mainframe. The existing SNA-based tool was quite complex and consumed more than 200 mainframe million instructions per second (MIPS). The new TCP/IP-based tool was faster, more accurate and consumed fewer than 3 mainframe MIPS. This reduction saves Merrill Lynch approximately $2 million per year in operating costs.
The success of this project, and the experience the team gained with TCP/IP, prompted the team to explore ways to expose this, and other mainframe data and services, to the distributed world. This led directly to the exploration of Web services standards and the development of the X4ML product.
Crew says the organization got a lot of credit from senior executives for the real-time quoting application, and just as important, the company's mainframe programmers recognized that Web services could give the mainframe new life in the Web world. It was during a period when Merrill was laying off some 24,000 workers globally, and mainframe programmers were chief among the casualties. "I went back and issued a challenge," Crew says. "I said, 'OK, we just saved $2 million; now let's go see if we can save the company $20 million.'"
In late 2000, Crew formed a strategy team to develop the Web services initiative. The group set out five requirements:
1. Mainframe programmers would not need to learn a new programming language like Java to create Web services.
2. The Web services development environment, where Web services would be built, analyzed and tested, would not require the purchase of software tools that had to be loaded onto powerful and expensive workstations. Instead, the group wanted development tools that could be accessed via a Web browser.
3. There had to be a centralized directory for listing Web services, so that programmers could reuse and combine services that had been developed.
4. Any Web service developed had to both maintain the security features that were already built into a mainframe application, and incorporate new Web security standards such as Web Services Policy Framework (WS-Policy) for areas including encryption, authentication and authorization.
5. The Web services architecture had to incorporate emerging Web services standards to allow future advancements.
Crew brought together a team that included programmers with experience in mainframe, Java and .NET environments to create the overall architecture for the Web services from scratch, as well as the development environment where Web services would be built, analyzed and tested.
The initiative was dubbed X4ML for "XML for Modernizing Legacy." Venkat Pillay, a database administrator who learned Java for the effort, says a large portion of the development time was spent strategizing what the architecture would look like. Of vital importance was that the platform not require changing application code on the mainframe or impede its performance in any way.
Those applications were tried and tested, and tinkering with mainframe applications can be like opening a can of worms, because of their complexity and the likelihood that the programmer and the knowledge behind the code are no longer around.