Purpose, Argument, Streams, Usage, Examples, Related

          ┌─STOP─ALLEOF────────┐
>>──DEAL──┼────────────────────┼─────────────────────────────────────────────────────>< 
          ├─STOP─┬─ALLEOF──┐   │
          │      ├─ANYEOF──┼───┤
          │      └─streams─┘   │
          │                    │
          ├─SECONDARY──────────┤
          │                    │
          │     ┌─1-*────────┐ │
          └─KEY─┼────────────┼─┘
                └─inputrange─┘

Use the DEAL stage to read records from its primary input stream and write records to one of its connected output streams in either sequential order; starting with the primary output stream, or some other order specified on the secondary input stream. Unless otherwise specified; DEAL writes the first input record to the connected primary output stream, the second input record to the connected secondary output stream, and so on, until the last connected output stream receives a record, and then the cycle is repeated.

When the SECONDARY keyword is specified; DEAL sequentially processes a secondary input record and primary input record as a pair. Each secondary input stream record must contain an integer output stream identifier which will control the order in which DEAL writes output records. In this way you can choose your own output stream order rather than use the sequential default. DEAL first reads and inspects a record from the secondary input stream, and retrieves the identity of the output stream that will receive a written record. DEAL then reads a record from its primary input stream. When DEAL determines that the output stream identified in the secondary input stream is connected; DEAL writes a record from the primary input stream to the specified output stream. DEAL then consumes the records from both its primary and secondary input streams.

You can also write a contiguous series of primary input stream records that have matching keys to the same output stream. DEAL searches for an identified location in each input record that contains the specified key, comparing the key from one record to the next. If the key location is not specified, or it is specified but is not found, DEAL uses the entire record as the key.

• STOP
  specifies how many output streams must be at end-of-file for DEAL to stop writing output records. STOP ALLEOF is the default.
  
  
  • ALLEOF
    specifies that input records will stop being written when DEAL determines that all output streams are no longer connected. Input records continue to be written as long as at least one output stream is connected. STOP ALLEOF is the default.
    
    
  • ANYEOF
    specifies that input records will stop being written when DEAL determines that all output streams are no longer connected. Input records continue to be written as long as at least one output stream is connected. STOP ALLEOF is the default.
    
    
  • streams
    is the number of streams required to reach end-of-file and become disconnected before DEAL stops writing records. Specify a decimal number of 1 or greater. Specifying 1 is the same as specifying ANYEOF.
    
    
• SECONDARY
  specifies that DEAL first reads and inspects a secondary input stream record containing a stream identifier before reading a primary input stream record and writing it to that identified output stream. One secondary input stream record is processed with one primary input stream record as a pair, so the order of the records on the secondary input stream controls the order in which DEAL writes primary input stream records to its output streams.
  
  
• KEY
  identifies the location in each input record that contains the key of each record. A contiguous group of input records with the same key are written to the same output stream. When the key changes from one input record to the next; DEAL writes the record with the changed key to the next sequential, connected output stream.
  
  
  • inputrange
    an integer column, word or field range that defines the key.

1. DEAL does not delay the records.
   
   
2. If the DEAL stage discovers that its primary or secondary input streams are not connected, the DEAL stage ends.
   
   If the SECONDARY or KEY keyword is specified and the DEAL stage discovers that the output stream it needs to write the next record to is not connected; DEAL ends and any remaining input records are not fully processed.
   
   If the DEAL stage discovers that the number of streams specified by the STOP keyword are not connected; DEAL ends. If streams is greater than the total number of output streams in the pipeline; DEAL processes as though STOP ALLEOF were specified.
   
   
3. Leaving an output stream unconnected will not cause records to be dropped; it causes DEAL to end.
   
   
4. With any keyword other than the SECONDARY keyword; DEAL verifies that its primary input stream is the only connected input stream and then begins execution. If the SECONDARY keyword is specified; DEAL verifies that both its primary and secondary input streams are connected and then begins execution.

1. The following example pipeline shows how you might use DEAL to control the order in which a particular output stream receives a record. By specifying the SECONDARY keyword on the DEAL stage; the pipeline writes input records to specific output streams defined by the LITERAL stage command in the third pipeline.
   
   

   **** Top of file ****
   Address Rxpipe
   
   'pipe (endchar ?)',
        literal /a j b p l/',          /* Input record data              */
       '| split',                      /* Split into individual records. */
       '| j: deal secondary',          /* Output stream numbers are on.. */
        ,                              /* ..the secondary input stream.  */
       '| specs /Stream 0:/ 1 1-* nw', /* Identify the output stream.    */
       '| k: faninany',                /* Read from any input stream     */
       '| console',                    /* Display on the console         */
       '?',
       'j:',
       '| specs /Stream 1:/ 1 1-* nw', /* Identify the output stream.    */
       '| k:',                         /* Route back to FANINANY.        */
       '?',
       'literal /0 1 0 1 1/',          /* Define output streams for DEAL */
       '| split',                      /* Split into individual records. */
       '| j:'                          /* Route back to DEAL on its..    */
                                       /* ..secondary input stream.      */
   Exit 0
   **** End of file ****
   

   
   

   output:
   
   Stream 0: a
   Stream 1: j
   Stream 0: b
   Stream 1: p
   Stream 1: l
   

   
   
2. Similar to the previous example; this next example also uses the SECONDARY keyword, with the notable difference that the target output stream/file number is extracted from the input file: 'salary.txt', at column 20, and is fed back into the DEAL stage on its secondary input stream. The pipeline produces three salary band output files: level1.txt, level2.txt and level3.txt.
   
   

   salary.txt
   
   ...|...+....1....+....2....+....3....+....4....
      **** Top of file ****
    1 IDENT. SALARY     BAND.   JOB TITLE
    2 ------ ---------- ------- ----------------
    3 000111  £4,000.00 L1      Associate
    4 011234 £12,000.00 L2      Staff
    5 096424  £8,000.00 L3      Senior Associate
    6 999990 £16,000.00 L2      Advisory
    7 111111 £99,000.00 L3      Senior
      **** End of file ****
   

   
   

   **** Top of file ****
   Address Rxpipe
   
   'pipe (endchar ?)',
        < salary.txt',        /* Input records containing employee info.   */
       '| drop 2',            /* Discard the two header records.           */
       '| f: fanout',         /* Copy input records to all output streams. */
       '| elastic',           /* Buffer output to avoid a pipeline stall.  */
       '| d: deal secondary', /* Secondary input has the deal order.       */
       '| console',           /* Stream 0 (primary) will have no records.  */
       '?',
       'f:',                  /* Secondary output stream connected.        */
       '| specs 20.1 1',      /* Level number in column 20 is stream ID.   */
       'd:',                  /* Secondary output stream connected.        */
       '| > level1.txt',      /* Write record to first output file.        */
       '?',
       'd:',                  /* Tertiary output stream connected.         */
       '| > level2.txt',      /* Write record to second output file.       */
       '?',
       'd:',                  /* Quaternary output stream connected.       */
       '| > level3.txt'       /* Write record to third output file.        */
   
   Exit 0
   **** End of file ****
   

   
   

   level1.txt
   
   ...|...+....1....+....2....+....3....+....4....       
      **** Top of file ****
    1 000111  £4,000.00 L1      Associate
      **** End of file ****
   

   
   

   level2.txt
   
   ...|...+....1....+....2....+....3....+....4....
      **** Top of file ****
    1 011234 £12,000.00 L2      Staff
    2 999990 £16,000.00 L2      Advisory
      **** End of file ****
   

   
   

   level3.txt
   
   ...|...+....1....+....2....+....3....+....4....
      **** Top of file ****
    1 096424  £8,000.00 L3      Senior Associate
    2 111111 £99,000.00 L3      Senior
      **** End of file ****
   

ELASTIC, FANIN, FANINANY, FANOUT