Merrill Lynch & Co.: Web Services, Millions of Transactions, All Good
Jim Crew could be considered an All-Star in the fledgling field of service oriented architecture. Over the past four years, Crew, as the former head of database infrastructure at Wall Street brokerage Merrill Lynch & Co., had spearheaded an effort to modernize the firm's multibillion-dollar investment in mainframe technology by building a platform to easily integrate many of the company's decades-old legacy applications with new software.
Since 2001, his team has helped generate as many as 420 Web services—essentially using a collection of Internet technologies like eXtenstible Markup Language (XML) and Simple Object Access Protocol (SOAP) to make operations currently performed on the mainframe available to programmers creating new applications in modern environments such as Sun Microsystems' Java or Microsoft's .NET.
In the process, he has helped Merrill Lynch save as much as $42 million in application development by being able to leverage business-hardened code on the mainframe and by making new hardware purchases unnecessary. In getting information directly from the mainframe and reusing existing legacy applications, Merrill has avoided having to copy information to more modern database platforms like Oracle and Sybase or rewriting software to run on application servers.
But much like skiing's Bode Miller, this all-star is also a rebel.
Unlike many other corporations that are embracing Web services, Merrill decided to do it without the help of a traditional vendor of Web services platforms, such as webMethods, Neon Systems or BEA Systems. Instead, it chose to develop and implement an entirely new Web services architecture on its own.
"We took a very careful look at what was available in the industry at the time. It would have been a lot easier to buy what we needed than build it," says Crew, who headed up Merrill's database organization in Hopewell, N.J. It would have also allowed Merrill to get up and running faster. "But we couldn't find anything that met our requirements," Crew explains. "In our opinion the [existing] SOA vendors were going about the problem the wrong way."
The crux of the problem, as Crew saw it, was that most of the Web services development platforms being pitched by the vendors had been built from the perspective of the distributed programmer, or programmers using modern development tools such as Java and .NET.
That just didn't make sense for Merrill because the firm would have to retrain many of its 1,200 programmers and purchase more powerful workstations to develop the Web services platform.
In fact, Merrill has a huge IBM mainframe installation—one of the largest in the world—with 1,200 programmers supporting some 23,000 mainframe programs that process 80 million Customer Information Control System (CICS) transactions per day. Each time a person logs on to a Merrill Lynch account online, or enters his password at a bank machine, could be considered one transaction, although the action of signing in at a bank machine or online would trigger many other transactions.
The company's mainframe programmers were the guardians of that "treasure." They knew what work was performed by the legacy programs and what information was stored in the databases, and how they might be leveraged by other applications in development. Training those programmers in Java or .NET would have been an expensive undertaking.
According to research firm Gartner, it can take an enterprise 12 months and as much as $67,000 to retrain a COBOL developer in Java.
In addition, it would have required Merrill to purchase new, more powerful desktops for its mainframe programmers to run the SOA application development software. As a result, Merrill opted to build its own Web development platform from scratch.
QUESTION: What's the biggest headache your tech team can expect to get from deploying Web services? Let us know at Baseline@ziffdavis.com
Story Guide:
Merrill Lynch & Co.: Web Services, Millions of Transactions;, All Good
Next page: Unlocking Access to Customer-Service Functions
-Service Functions">
The actual case for developing Web services was a no-brainer. As reliable and robust as Merrill's mainframe infrastructure was, it was also fraught with limitations. As new Internet-based applications, such as self-help credit card balance checks, were being developed, programmers needed to access data locked in Merrill's mainframe vaults. It was difficult to tap directly into those vaults using non-mainframe-based software.
In fact, it's estimated that 90% of all new development costs involve the plumbing to integrate mainframe applications with newer platforms. Merrill had instead taken the approach of copying that data into Oracle, Sybase or Microsoft SQL databases, which could more easily integrate with server-based applications.
But the copying of the data created two problems: Copying tends to be unreliable because of disk errors, read errors and running-out-of-space errors. Second, the data could be out-of-date as soon as it is copied.
For example, a client making several trades would have to wait until the following day to see an accurate balance in his account. As a result, a client might make a trade believing there were adequate funds, but that trade would have to be rejected because, in fact, the funds were not available.
"A lot of money was being spent copying data to Oracle or SQL servers, and the frustrating thing was this information was available and up-to-date on the mainframe—it was just difficult to get at," Crew says.
Web services seemed the ideal solution to the challenge. Through a set of widely accepted standards, such as XML, which uses tags to describe data and which can in turn be understood by any XML-aware application; and SOAP, which provides a means for a program in one kind of operating system to communicate with a program in another kind of operating system, it was possible to open up Merrill's mainframe vault to modern applications.
In a simple scenario, a client using an online application to look up his credit card balance submits a request via a Web browser. The mainframe runs the requested operation (obtain credit card balance) and sends the information back to the client via SOAP. XML makes it easier for the mainframe to interpret the online application, and in turn assists in allowing the online program to understand the response delivered by the mainframe.
In 2000, Crew formed a team within the database group to begin exploring the relatively new CICS Transport Control Protocol/Internet Protocol (TCP/IP) capabilities. As a result, the group wrote a tool, using CICS TCP/IP, to replace an existing mainframe tool used for getting real-time market data to the mainframe.
The existing tool employed the more traditional Systems Network Architecture (SNA) protocols, a set of network protocols developed by IBM, and was used to allow a Merrill Lynch customer accessing his account online to get pricing of a traded security directly from the mainframe. The existing SNA-based tool was quite complex and consumed more than 200 mainframe million instructions per second (MIPS). The new TCP/IP-based tool was faster, more accurate and consumed fewer than 3 mainframe MIPS. This reduction saves Merrill Lynch approximately $2 million per year in operating costs.
The success of this project, and the experience the team gained with TCP/IP, prompted the team to explore ways to expose this, and other mainframe data and services, to the distributed world. This led directly to the exploration of Web services standards and the development of the X4ML product.
Crew says the organization got a lot of credit from senior executives for the real-time quoting application, and just as important, the company's mainframe programmers recognized that Web services could give the mainframe new life in the Web world. It was during a period when Merrill was laying off some 24,000 workers globally, and mainframe programmers were chief among the casualties. "I went back and issued a challenge," Crew says. "I said, 'OK, we just saved $2 million; now let's go see if we can save the company $20 million.'"
In late 2000, Crew formed a strategy team to develop the Web services initiative. The group set out five requirements:
1. Mainframe programmers would not need to learn a new programming language like Java to create Web services.
2. The Web services development environment, where Web services would be built, analyzed and tested, would not require the purchase of software tools that had to be loaded onto powerful and expensive workstations. Instead, the group wanted development tools that could be accessed via a Web browser.
3. There had to be a centralized directory for listing Web services, so that programmers could reuse and combine services that had been developed.
4. Any Web service developed had to both maintain the security features that were already built into a mainframe application, and incorporate new Web security standards such as Web Services Policy Framework (WS-Policy) for areas including encryption, authentication and authorization.
5. The Web services architecture had to incorporate emerging Web services standards to allow future advancements.
Crew brought together a team that included programmers with experience in mainframe, Java and .NET environments to create the overall architecture for the Web services from scratch, as well as the development environment where Web services would be built, analyzed and tested.
The initiative was dubbed X4ML for "XML for Modernizing Legacy." Venkat Pillay, a database administrator who learned Java for the effort, says a large portion of the development time was spent strategizing what the architecture would look like. Of vital importance was that the platform not require changing application code on the mainframe or impede its performance in any way.
Those applications were tried and tested, and tinkering with mainframe applications can be like opening a can of worms, because of their complexity and the likelihood that the programmer and the knowledge behind the code are no longer around.
Story Guide:
Merrill Lynch & Co.: Web Services, Millions of Transactions;, All Good
Next page: Despite Complexity, Merrill Skipped Middleware
Middleware">
The team considered using a middleware server for the key job of translating requests in one language, such as Java, into instructions that could be understood by the mainframe, but opted to implement the translation piece directly on the mainframe. The reason: speed and simplicity. Using fewer moving parts—by eliminating the server—means higher reliability and writing the translation code in Assembler, a programming language for the mainframe, delivering higher performance than a Java parser.
The X4ML development environment where Web services are created, analyzed and tested sits on a Java server—not on a mainframe—and was written in Java 2 Enterprise Edition, which was selected because of its ability to work with multiplatform architectures. The Java server can be accessed by any browser-equipped computer, satisfying the requirement to allow Merrill's mainframe programmers to develop Web services without the need for high-end workstations.
The key thing, Crew says, is that there is no middleware between the requesting application and the mainframe. For example, a Merrill financial adviser may submit a request from his desktop to find all of his clients with shares in ExxonMobil, perhaps due to a sudden dip in the stock. The request is submitted from the adviser's desktop application, and the search is run directly on the mainframe via a SOAP request. SOAP tells the mainframe which operation to perform, and XML does the translating.
The X4ML Web services development environment is where the magic is created. A mainframe programmer accesses the X4ML tool via his desktop browser. He assigns the new service a name, such as "Client Finder," then tells the tool where to find the application by providing its mainframe listing.
The programmer then clicks on an Import button, and X4ML learns the program's "signature"—that is, the program's variables and how they are used.
Next, the programmer uses an analyzer tool to essentially pull the program apart and determine each of its inputs and outputs. By doing this, a programmer can create a Web service tailored to perform all or just parts of an operation contained in a legacy application. For example, a client finder application on the mainframe may produce results that list the name, address, phone number, birthdate and Social Security number of a client.
However, the programmer may want to produce a Web service that just retrieves name and address but not the other data, due to privacy concerns. Increased security policies, such as requiring authentication or encryption, can also be assigned. Alternatively, the programmer can click on a One Click button to automatically create a Web service based on the signature of the program that was determined by the Import tool.
Either option creates the Web service and automatically lists the new service in a Universal Description, Discovery and Integration (USSI) directory, where it can be accessed and utilized by other programmers.
A testing function within the X4ML tool also lets the programmer test the operation and evaluate errors before it is used. Creating new applications from the Web services then becomes a matter of accessing and combining operations in the directory.
The value of Web services was demonstrated over the last three years when Merrill launched a $1 billion effort to create a new suite of applications for wealth management that are available for use by Merrill's 14,000 financial advisers. The dual-screen desktop essentially provides these advisers with access to research, tools and account information to serve clients.
The wealth management project involved the creation of a wide range of new programs, in partnership with Thomson Financial, which brought its range of financial data and research services to the table, and Siebel Systems, for its client management expertise. Where possible, Merrill leveraged functions and data already available on the mainframe.
There was no need to create a complicated system that would allow a financial adviser to search through 1,200 clients to find just the ones owning ExxonMobil shares, when such a search could easily be run on the mainframe and be enacted by a Web service.
QUESTION: What's the biggest headache your tech team can expect to get from deploying Web services? Let us know at Baseline@ziffdavis.com
Story Guide:
Merrill Lynch & Co.: Web Services, Millions of Transactions;, All Good
Next page: Mainframe Remains Strategic
Andrew Brown, Merrill's chief technology architect, says what shouldn't be lost in the excitement around Web services is what the technology means for mainframe applications overall.
"We are making the mainframe strategic to our future. It's a fundamental decision," he says. "Back in the 1990s, it was trendy to say you were migrating off the mainframe. Now, I couldn't predict when the mainframe will go away. If anything, technologies like X4ML have given it new life."
In December, Merrill decided to give X4ML a chance to prove itself in the larger marketplace. The technology was sold for an undisclosed sum to SOA Software, a Los Angeles-based vendor of service oriented architecture systems. Four members of the development team—Crew; Michael Card, a database administrator and one of the chief developers of X4ML; Pillay; and Bruno Vitro, a Java expert who was hired to help develop X4ML—have transferred to SOA Software to shepherd the continued development. X4ML has since been renamed Service Oriented Legacy Architecture (SOLA).
According to Brown, Merrill has a long history of selling off technology that it has developed internally, and that it feels will not threaten its competitive advantage. In return, Brown hopes SOA will be able to further enhance the newly named SOLA, particularly in the area of registry services and by continuing to adopt and implement new security standards.
"We spend a lot of money on technology—billions of dollars a year," he says. "So, we're always looking for ways we can optimize that investment."
For now, Crew and the rest of the X4ML team continue to work from Merrill's Hopewell campus, not far from the stately grounds of Princeton University. Looking out his third-floor window, across a still-unfrozen pond toward Merrill's credit card operations, Crew feels a sense of satisfaction and excitement about the future.
"In many ways, this is the culmination of a life's work," he says. "People like myself have spent their entire careers building mainframe applications, testing and refining them, making them work faster and absolutely reliably. Mainframers see this [Web services] as a way to leverage and build on all of that hard work."
QUESTION: What's the biggest headache your tech team can expect to get from deploying Web services? Let us know at Baseline@ziffdavis.com
Story Guide:
Merrill Lynch & Co.: Web Services, Millions of Transactions;, All Good
Next page: Base Case: Merrill Lynch By the Numbers
: Merrill Lynch By the Numbers">
Merrill Lynch & Co. Base Case
Headquarters: 4 World Financial Center, New York, NY 10080Phone: (212) 449-1000
Business: Provides an array of investment banking, financing, wealth management, insurance and related services on a global basis.
Chief Technology Architect: Andrew Brown
Financials in 2005: $47.8 billion in revenue, $5.2 billion in profits.
Challenge: Leverage the company's heavy investment in mainframe applications and hardware by making functions in legacy applications available as Web services to other applications.
BASELINE GOALS:
|
|
QUESTION: What's the biggest headache your tech team can expect to get from deploying Web services? Let us know at Baseline@ziffdavis.com
Story Guide:
Merrill Lynch & Co.: Web Services, Millions of Transactions;, All Good
