Using IBM IPCS to battle software bugs

If you experience software bugs on CICS, you are likely familiar with IBM's interactive problem control system (IPCS). The command functions work, but not efficiently. Robert Crawford explains how to make IPCS work for you.

Those of us who have software bugs use IBM's interactive problem control system (IPCS) nearly every day. The ISPF browse panels (option 1) make it easy to look at the dump's contents and assign labels. The command interface (option 6) allows you to enter IPCS line commands including extensions through the use of verb exits supplied by the major subsystems (e.g., CICS, IMS, MQ etc.) to format their otherwise object code only (OCO) control...


As wonderful as IPCS is, there are still a couple problems. First, IPCS commands and verb exits deal with specific system components with little integration between them. Second, debugging often involves repetitive, error prone activities such as control-block chaining. Not only is it a boring activity, you have to keep careful notes to remember where you are. Fortunately, the IPCS command interface also allows users to write command lists (CLIST's), Rexx programs and ISPF dialogs to get around these shortcomings.

Extending the IPCS command interface
You invoke user written scripts from IPCS line command panel. IPCS commands embedded in the scripts can list, extract and analyze dump data, thus presenting an opportunity to automate the more mechanical debugging tasks as well as create custom analysis routines tailored to a particular system's or application's structure. Debugging scripts may also trap IPCS command output and use the data from one system component in a command for another. The most useful IPCS script commands are LIST, NOTE and EVALUATE.

LIST and NOTE write output to the IPCS log. Inside of a Rexx program or CLIST, LIST works just as it does from the IPCS command line; you enter LIST followed by an address or symbol and an optional length. NOTE, on the other hand, allows you to write your own messages to IPCS' output stream. This is useful for error messages or English interpretation of system control blocks. Evaluate (abbreviated EVAL) inserts dump contents into a CLIST, Rexx or ISPF dialog variable. Once inside the variable the data may be used for analysis or as input for another command.

Example of DSALIST Rexx program
IBM's Language Environment (LE) provides several handy verb exits for formatting its control blocks. Unfortunately I often run into situations where an application wrecks the stack frame so badly that LE can't format the traceback to list the program call sequence.

LE creates the traceback starting with register 13 (R13) and linking backwards through dynamic storage areas (DSA). If R13's contents are invalid or a clobbered DSA breaks the chain, the next best option is to start at the beginning and work forward.

LE's mother block for each thread is the common anchor area (CAA). Offset +2E0 points to the dummy DSA (DDSA) at the beginning of the DSA chain. At offset +4C of each DSA is the next available byte (NAB) pointer which LE uses to create the next stack upon entering a new subroutine. In a dump you can use the NAB to chain from the DDSA to each subsequent DSA until the backward save area pointer (+4 in every DSA) doesn't match the previous DSA address.

This is easily done manually, although it becomes somewhat tedious when the call stack grows to be more than 10 or so routines. That's where the DSALIST Rexx program, shown below, comes in handy.

/* REXX */

parse arg caa
address ipcs /* A */
if caa = "" then,

"note 'You must supply the CAA address'"
"note 'CAA address:" caa"'" /* B */

"equ caa" caa /* C */
"evaluate" caa"+2e0 rexx(storage(ddsa)) length(4)" /* D */
"note 'dummy dsa address:" ddsa"'"
"equ ddsa" ddsa
backChain = ddsa

thisDSA = ddsa
oldDSA = ddsa
DSALabel = "ddsa"
i = 1
do while(oldDSA = backChain)
"list" DSALabel "length(128)" /* D */
oldDSA = thisDSA
"evaluate" DSALabel"+4c rexx(storage(thisDSA)) length(4)"
DSALabel = "DSA" || i
"equ" DSALabel thisDSA
"evaluate" DSALabel"+4 rexx(storage(backChain)) length(4)"
i = i + 1
"note 'End of DSA chain reached'"

The script's one input parameter is the CAA address. You can usually find the CAA address in register 12 (R12) or at offset +94 in the CICS system task control area (STCA). If neither address is good, you can also search the dump for the literal "CEECAA." This eyecatcher appears at offset -18 from the beginning of the CAA.

The more interesting lines in this program are labeled with comments in bold:

  1. As in all Rexx programs, you must use the ADDRESS verb to tell it who will execute the commands. Here we must send them to IPCS.
  2. The IPCS NOTE command prints data in the IPCS log. This line shows that literals and variables can be mixed.
  3. Here an IPCS EQU (equate) command assigns the label "CAA" to the address provided as an input parameter. After this assignment you can use this label in storage references outside of this script.
  4. This line uses the EVALUATE command to store dump data into Rexx variable DDSA. Note that standard IPCS storage references apply. In this case the command stores the four bytes at offset +2E0 from label CAA in Rexx variable DDSA
  5. This IPCS LIST command will print the contents of the current DSA. Note the length field is decimal.

The program runs a loop that links through the DSAs using NAB pointers. It escapes from the loop when the backward save area pointer at +4 into the DSA no longer matches the previous DSA address.

The output from this Rexx is too awkward to show here. However, it is only part of the story as the LEDATA verb exit can also produce a traceback using a DSA address. Therefore, if "cccccccc" is the CAA address and "dddddddd" is the address of the last DSA noted in DSALIST, this command:

"VERBX LEDATA 'CAA(cccccccc),DSA(dddddddd),ALL'
will (hopefully) print a traceback of the routines leading up to the stack error. If the traceback doesn't work you may still get some useful information out of the register save area (RSA) in the first 72 bytes of each DSA.

Assembler quiz
Here's an Assembler question for you: What commonly used instruction can add three numbers together and put the sum in a fourth location? Send your answers to The first three correct answers received will win a $5 Starbucks card. I'll disclose the solution next month.

ABOUT THE AUTHOR: For 24 years, Robert Crawford has worked off and on as a CICS systems programmer. He is experienced in debugging and tuning applications and has written in COBOL, Assembler and C++ using VSAM, DLI and DB2.

This was first published in October 2008

Dig deeper on Mainframe operating systems and management



Enjoy the benefits of Pro+ membership, learn more and join.



Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: