FILEOUT stage

Purpose, Argument, Streams, Usage, Examples, Related

Syntax

                                      (2)┌─WRITE──┐ 
                 ┌─NOCLOse─┐             ├─R/W────┤
>>──┬─>───────┬──┼─────────┼──┬───────┬──┼────────┼──┬──────────┬────────────────────>< 
    └─FILEOUT─┘  └─CLOse───┘  └─RECno─┘  ├─RDONLY─┤  └─filename─┘
                                         └─R/O────┘    (1)

Notes:
(1) The filename operand is only allowed if the stage does not have a secondary 
    input stream connected.
(2) Unless the WRITE keyword is specified; a file that is write-protected
    (RDONLY) remains write-protected after it has been processed.
(3) FILEOUT cannot be specified as the first stage in a pipeline.

Purpose

Use the FILEOUT stage to create a disk file or replace the contents of a disk file.

FILEOUT reads records from its primary input stream and writes them to the specified file and to its primary output stream, if it is connected. FILEOUT appends a Line-Feed character to each output record that it writes to the specified file, primary output stream records are written unchanged. If the specified file does not already exist; FILEOUT creates a new file containing the records read from its primary input stream. If the specified file already exists, FILEOUT replaces the contents of the file with the records read from its primary input stream.

If you do not specify a fully qualified path and filename; the FILEOUT stage will create a new file in the current working directory.

Argument

NOCLOse
specifies that the output file is closed once a new output filename is encountered.
CLOse
specifies that the output file remains open until end-of-file is reached on the primary input stream.
RECno
specifies that a record number should be inserted in to both the output file record and the primary output stream record.
WRITE
R/W
specifies that the output file will be enabled for reading and writing. Note, that when FILEOUT writes to an existing, write-protected file; unless you specify the WRITE keyword, by default FILEOUT will save the file as write-protected.
RDONLY
R/O
specifies that the output file will be write protected. The file's arttibute will be set to read-only. Note, that when FILEOUT writes to an existing, write-protected file; unless you sepcify the WRITE keyword, by default FILEOUT will save the file as write-protected.
filename
is the name of the output file.

Streams

The following streams and files are used by the FILEOUT stage:

Input streams
- When the secondary input stream is connected
  
  FILEOUT will not allow the filename argument to be specified. FILEOUT first reads a record from its secondary input stream, which it treats as the name of the file to which it should write.
  - With the CLOse keyword; if the content of the record (the filename) is different from the previous secondary input stream record, FILEOUT closes the current output file and opens the new one specified in the record.
  - With the NOCLOse keyword (the default); if the content of the record (the filename ) is different from the previous secondary input stream record, FILEOUT first searches the file cache for an entry matching this new file and if one exists, FILEOUT simply switches its focus to this existing (already open) file. Otherwise, FILEOUT opens the new file and adds its details to the file cache.
  FILEOUT next reads a record from its primary input stream; writes this record to the target output file and then writes the record to its primary output stream, if it is connected. This process continues until the end-of-file is reached on the primary input stream. At end-of-file; FILEOUT traverses the cache and closes all of the files.
  - Secondary input stream:
```
input:

|...+....1....+....2....+....3....+....4....
c:\myfile.dat
c:\myfile.dat
d:\yourfiles\yourfile.dat
..
..
d:\yourfiles\yourfile.dat
```
  - Primary input stream:
```
input:

|...+....1....+....2....+....3....+....4....
record
record
record
..
..
record
```
- When the secondary input stream is not connected, the filename argument is allowed
  1. When the filename argument is specified
    
    FILEOUT reads records from its primary input stream and writes them unchanged to the specified file, and to its primary output stream, if it is connected.
    
    Primary input stream:
```
input:

|...+....1....+....2....+....3....+....4....
record
record
record
..
..
record
```
  2. When the filename argument is not specified.
    
    FILEOUT expects its primary input stream records to follow a specific format; the record must comprise an output filename, followed by the delimiter 0xff and the record (data) to write to the output file.
    
    FILEOUT extracts the output filename and if it differs from the filename specified in the previous primary input stream record (the current output file), and,
    - With the CLOSe keyword; if the content of the record (the filename) is different from the previous primary input stream record, FILEOUT closes the current output file and opens the new one specified in the record.
    - With the NOCLOse keyword (the default); if the content of the record (the filename) is different from the previous primary input stream record, FILEOUT first searches the file cache for an entry matching this new file and if one exists, FILEOUT simply switches its focus to this existing (already open) file. Otherwise, FILEOUT opens the new file and adds its details to the file cache.
    FILEOUT then writes the record (data) part of the primary input stream record to the specified file, and to the primary output stream, if it is connected.
    
    Primary input stream:
```
input:

|...+....1....+....2....+....3....+....4....
c:\myfile.datXrecord
c:\myfile.datXrecord
d:\yourfiles\yourfile.datXrecord
..
..
d:\yourfiles\yourfile.datXrecord
```
Output file
- Without the RECno keyword:
  
  FILEOUT writes the data part of the primary input stream record to the specified file.
```
output file:

...|...+....1....+....2....+....3....+....4....
   **** Top of file ****
 1 record
 2 record
 3 record
 . ..
 . ..
99 record
   **** End of file ****
```
- With the RECno keyword:
  
  FILEOUT writes the current output record number; a ten-digit field, right-aligned with leading spaces, the delimiter 0x20 (a blank) and the contents of the primary input stream record to the specified file.
```
output file:

...|...+....1....+....2....+....3....+....4....
   **** Top of file ****
 1          1Brecord
 2          2Brecord
 3          3Brecord
 .         ..
 .         ..
99         99Brecord
   **** End of file ****
```
Output streams:
- Primary output stream:
  (without the RECno keyword)
  
  FILEOUT writes the data part of the primary input stream record to the specified file.
```
output:

|...+....1....+....2....+....3....+....4....
record
record
record
..
..
record
```
- Primary output stream:
  (with the RECno keyword)
  
  FILEOUT writes the current output record number; a ten-digit field, right-aligned with leading spaces, the delimiter 0x20 (a blank) and the contents of the primary input stream record to the specified file.
```
output:

|...+....1....+....2....+....3....+....4....
         1Brecord
         2Brecord
         3Brecord
        ..
        ..
        99Brecord
```

Usage

FILEOUT does not delay the records.
If FILEOUT discovers that its primary input stream is not connected, the FILEOUT stage ends.
FILEOUT determines if an output exists at the moment the file is opened for writing. FILEOUT opens an output file for writing when there is an output record to write. FILEOUT does not create empty files.
When FILEOUT opens an existing output file for writing; the contents of the file, if any, are replaced.
FILEOUT appends a Line-Feed character to all but the last record that it writes to the specified output file.
FILEOUT does not write directly to the target output file; to preserve the integrity of the target file (an existing file) in the event of a pipeline runtime error (or a user intervention – CTRL-C, end-task or close-window); FILEOUT writes its output records to an intermediate work file. Only at the end-of-file on the primary input stream will FILEOUT swap this work file for the final target output file.

When the FILEOUT stage is interrupted or if FILEOUT encounters an unrecoverable error (and the stage may be unable to finalise its final file the file-write) the intermendiate work file(s) will be left as-is and available for inspection.

A work file has a specific filename format that will allow you to determine what the file was created for, the manager, and pipeline and stage that created it. The following schema defines the format of a work file:
```
The format of an intermediate work file is:

FILEOUT.TYPE=%s.FILE=%s.PID=%.5d.PIPE=%.4d.STAGE=%.4d.UNQ=%10.0d.DAT

Element      Meaning
-------      -------

%s           The stage (DLL name) FILEOUT
.            separator
TYPE=%s      'NEW' or 'SWAP'
.            separator
FILE=%s      The target output filename (including .ext if it is specified)
.            separator
PID=%.5d     The process identification number of the manager
.            separator
PIPE=%.4d    The pipeline number
.            separator
STAGE=%.4d   The stage number
.            separator
UNQ=%10.0d   A ten digit (unique) number derived from the current system time
.            separator
DAT          The file extension
```
FILEOUT verifies that its secondary output stream is not connected and then begins execution.

Examples

The following pipeline reads records from the console and writes them to the file: myfile.txt. The pipeline ends when a null record is entered.
```
'pipe console | > myfile.txt 
```

The following example show how the CONSOLE stage reads interactively from the keyboard; writing each of the input records to the output file: keyboard.txt. For each input record; the FILEOUT stage produces an output record that comprises a ten-digit field, right-aligned with leading spaces, the delimiter 0x20 (a blank) followed by the contents of the input record.

**** Top of file ****
Address Rxpipe

'pipe cons eof /stop/',
    '| xlate',
    '| recno >',
    '| console'

Exit 0
**** End of file ****

console:

CONSOLE(0): 'stop' to end.
CONSOLE(0): ?
one
         1 ONE
CONSOLE(0): ?
flew over
         2 FLEW OVER
CONSOLE(0): ?
the cuckoos
         3 THE CUCKOO'S
CONSOLE(0): ?
nest
         4 NEST
CONSOLE(0): ?
stop

keyboard.txt

...|...+....1....+....2....+....3....+....4....
   **** Top of file ****
 1          1 ONE
 2          2 FLEW OVER
 3          3 THE CUCKOO'S
 4          4 NEST
   **** End of file ****

CONSOLE, >> (FILEAPPEND), > (FILEIN), IN, OUT

History

Version

Date

Action

Description

Pipelines

1.1

??.??.2025

changed

Application-wide rewrite

2.1

1.0

06.09.2007

created

First version.

1.0