Gotcha! Deploying Job-Scheduling Software

By David F. Carr  |  Posted 2003-06-01 Print this article Print

Despite all the talk about "real-time" systems, batch scheduling is often the order of the day.

Did you know that:

  • Despite all the talk about "real-time" systems, batch scheduling is often the order of the day

    Companies need software programs, called job schedulers, to control what programs are going to run and when, to keep orders handled and operations humming.

    But this doesn't necessarily happen the moment an order is placed.

    "There's a misconception that just because the real-time enterprise is here, all processes will go real-time," says Gartner Inc. analyst Milind Govekar. "But batch has been around since about day two in computing, and it's not going away."

    To minimize use of its network, a company may send orders in batches at set times to a distribution center.

    At Eckerd, updates to its data warehouse—its central data repository—are scheduled to run nightly, after stores have reported their sales for the day. Eckerd uses IBM's Tivoli Workload Scheduler, but there are other job schedulers, such as BMC's Control-M, that also can ensure each job is handled successfully.

  • Just because a process is automated doesn't mean it's automated efficiently

    For example, existing job schedules required too much "hand-holding," according to Eckerd officials, due to "dependencies" between jobs. Dependencies exist wherever one job must be completed before the next job can begin. Such dependencies tend to accumulate as new jobs are programmed into a scheduler over time and it can be easy to lose track of them.

    At Eckerd, sales data sometimes wasn't sent to the data warehouse for two or three days because the process of gathering and consolidating sales data frequently stalled and had to be restarted by data-center computer operators. Job scheduling software works by launching multiple programs or batch scripts in the correct order to achieve an overall result—in this case, an update of the data warehouse. If the program for processing pharmacy insurance claims fails, for example, pharmacy sales totals won't be available to feed into the overall sales results.

    Eckerd's software team programmed the scheduler to log when a job took longer than expected. This allowed Eckerd to analyze the dependencies that were causing the process to break down and concentrate on solving those problems. Today, Eckerd's data warehouse is updated every night.

  • Multiple schedulers mean complications

    If multiple administrators are doing their own scheduling at different sites or on different installations of the software, tracking dependencies is even harder.

    Steve Farrell, an operations manager in Merrill Lynch's mainframe data centers, recalls that, a few years ago, the broker had three mainframes with their processing controlled by separate organizations. A dependency breakdown "did stop one data center in its tracks waiting on a feed from another."

    The incident taught Merrill's data center managers they needed to do a better job of documenting the relationships between jobs running at different sites, says Farrell.

    David F. Carr David F. Carr is the Technology Editor for Baseline Magazine, a Ziff Davis publication focused on information technology and its management, with an emphasis on measurable, bottom-line results. He wrote two of Baseline's cover stories focused on the role of technology in disaster recovery, one focused on the response to the tsunami in Indonesia and another on the City of New Orleans after Hurricane Katrina.David has been the author or co-author of many Baseline Case Dissections on corporate technology successes and failures (such as the role of Kmart's inept supply chain implementation in its decline versus Wal-Mart or the successful use of technology to create new market opportunities for office furniture maker Herman Miller). He has also written about the FAA's halting attempts to modernize air traffic control, and in 2003 he traveled to Sierra Leone and Liberia to report on the role of technology in United Nations peacekeeping.David joined Baseline prior to the launch of the magazine in 2001 and helped define popular elements of the magazine such as Gotcha!, which offers cautionary tales about technology pitfalls and how to avoid them.

    Submit a Comment

    Loading Comments...
    eWeek eWeek

    Have the latest technology news and resources emailed to you everyday.