Pipes vs Pipelines
Sometimes one hardly recognizes one's own children! CMS Pipelines was inspired by Unix pipes but morphed into something recognizably different under the CMS guest operating system running under IBM VM .
Once you get used to the idea that topologies need not be simple left-to-right, there's a lot of things that can be done in a very natural efficient way. - Jeff Savit
The theoretical difference between Pipelines and Unix pipes is exemplary of the difference between the Unix viewpoint and the Mainframe viewpoint. On Unix, data goes from Here to There. On Mainframes, data goes from Lotsa Places to Everywhere.
When Roland Mainz asked me on the SOL-390 list:
Jack Woehr wrote:<br />
<span class="moz-txt-citetags">> </span>The main coolosity about Pipelines vs Unix pipes (from a Unicist point<br /><span class="moz-txt-citetags">> </span>of view) is that Pipelines are inherently "tees".<br />
<!---->Can you describe this from an Unix standpoint somehow, e.g. a short<br />pseudocode example would be nice...
Jeff Savit, a Pricinpal Field Technologist at Sun Microsystems, jumped in with the following. It's quoted with Jeff's kind permission:
I'll describe with actual code. Here's a trivial example. In VM, the command "QUERY NAMES" (something like the Unix command "who", and can be abbreviated to "Q N", and typically is entered in lower case from the terminal) yields output like:
LOGL0022 -L0022, MAINT4 -L0007, MONWRITE - DSC , EREP - DSC
DISKACNT - DSC , VMBACKUP - DSC , VMTAPE - DSC , VMSCHED - DSC
TCPVSW1A - DSC , ZIP05LNX - DSC , VM - 00C0, ZIP00SOL - DSC
ZIPK - DSC , ZIPL - DSC , ZIPA - DSC , SMTP - DSC
NAMESRV - DSC , FTPSERVE - DSC , UFTD - DSC , REXECD - DSC
TCPIP - DSC , ISPVM - DSC , VSM - DSC , PERFSVM - DSC
VMPRF - DSC , VMSECURE - DSC , OPERATOR - DSC , V102812 -L000B
VSM - TCPIP
Given this, I'd like to see how many users are logged on at a terminal, or how many are service virtual machines (the VM equivalent of a daemon), and how many sessions are at a terminal. It's easy to issue a command, see who is at the "not really a terminal" identifier "DSC" (for "disconnected"), and filter out any special userid marked as a "VTAM service machine" (a misnomer actually) that creates a logical terminal, and is marked by the tag 'VSM'. So a pipe could look like:
/* Sample pipe */
'PIPE (end ?) cp query names',
'| split at ,',
'| a: locate word 3 /DSC/',
'| count lines',
'| spec word 1 1 /disconnected users/ nextw',
'| nlocate word 1 /VSM/',
'| count lines',
'| spec word 1 1 /Connected users/ nextw',
and generate the following output:
4 Connected users
24 disconnected users
Notice several things:
- the "(end ?)" pieces specifies the character that marks the end of a pipe segment
- the cp stage issues the CP command and retrieves its output, passing it to its standard output
- the split command spits the incoming text at the marked item
- the locate command matches all the lines containing the specified string (here, I require it be in the 3rd word of the line)
- lines containing the desired string go to standard output, where I count the number (like a "wc -l") and then format an output line.
- "spec" is the all-purpose text formatting pipe stage with lots of features for formatting and selecting fields.
- lines not comtaining that "DSC" are sent to a pipe segment with the label "a:", which works in similar manner.
- the generalization is that a stage can have any number of input streams and output streams (primary, secondary, tertiary)
- stages like "count" are "blocking stages", since they delay putting an output record till they consume all their input records.When the primary input on the count stage is closed (end of input), then count emits its record.
- stages like "locate", "nlocate" ("NOT" applied to locate) do not block, but pass data to its primary or secondary output as needed.
So, the locate stage has a primary output stream ("matched") and a secondary stream ("unmatched"), and I can direct that to another part of the pipe for different processing.
This is just a trivial case. Once you get used to the idea that topologies need not be simple left-to-right, there's a lot of things that can be done in a very natural efficient way.