|
History of change |
Pipelines |
|
This history of change document is intended to provide a synopsis of
the notable changes that have been made between versions of Pipelines. It
will detail only those changes which are significant, specifically; changes which
may affect the functionality of existing, user-written pipelines. |
Matrix
|
|
Version
2.1 |
Pipelines 2.1 (32bit and 64bit)
for ooRexx version 4+ will contain the following changes:
|
● |
A
new .INI file will be introduced which will define many of the internal
Pipelines settings and values. This will allow the user to customise
Pipelines to a much greater extent than is currently possible. (This also
means that editing the registry in order to modify some of the current
settings will no longer be necessary. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
CApplMessage
class API function, IssueMessage(); the console
message mechanism that writes ALL Pipelines output to the physical screen,
will be changed. IssueMessage() will portion its
output into blocks; writing each block to the console, one after another.
This will provide better memory management for very large output message.
(Consider the output generated by the TRACE mechanism; which may write input
and output records of considerable size to the console). |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
The
following changes will be made to the canned message tables:
|
||||||||||||||||||||||||||||||||||||||||||||
|
● |
The
format of the TRACE console output will be changed; with the hexadecimal representation
of a record being displayed vertically instead of horizontally (this will
reduce both the ‘Data:’ and ‘Hex:’ display output length by half). In
addition; a new output line ‘Text:’ will be added that will be formatted
using an emulated tab-stop length of a definable length. Document: StageManager.htm will be updated to
reflect this change. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
CONSOLE
stage command: The DELIMPAIR operand will be renamed to DELIMIT and the prefix
and append delimiter operands will be re-defined as character strings. In
addition; CONSOLE will no longer use these delimiters when the pipeline or
stage is running in trace mode, it will instead, format an output record for
display on the console (the output record written to the primary output
stream will continue to remain unchanged) using an emulated tab-stop length
of 8 characters. Document: Console.htm
will be updated to reflect these changes. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
DELAY stage
command: The WAIT and NOWAIT operands will be removed. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
The following
CStageManager class API functions will be renamed:
API header: CStageManager.h
and API documents: CStageManager.htm
and others (that cross-link from example code) will be updated to reflect
this change. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
FILEAPPEND
(>>) stage command: is incorrectly adding an initial blank/null line on
its output to each new file it switches to when handling multiple output
files. This will be corrected. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
SHELLEXECUTE
stage command: the existing BATCH operand will be removed and a new
WARNING/NOWARNING operand will be introduced; which will allow the
specification of an ‘up to and including’ maximum acceptable process return
code. The NOWARNING operand will allow the stage to execute commands without
generating either warning or error messages for return codes that are less
than that defined by the MAXERROR errorlevel
operand. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
STEM stage
command: incorrectly requires a primary input stream to be connected. This
will be corrected. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
DELAY stage
command: incorrectly requires a primary output stream to be connected. This
will be corrected. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
The ‘Substitution
placeholder’ mechanism will be renamed to ‘Macro’ and will be changed to
support a new syntax specification: [¯o] where; macro can be any one of the following
definitions:
The existing
‘Substitution placeholder’ macros will be no longer be
supported. Document: Pipelines.htm
will be updated to reflect this change. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
A re-write of the
install package to handle UAC access restrictions on the system registry. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
A full review and
update of the online website documentation package and install ooRexx
examples. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
LOOKUP stage
command: is incorrectly writing a master record followed by a matching detail
record to the primary output stream by default. This will be corrected; to
write a detail record followed by a matching master record to the primary
output stream. LOOKUP is not correctly matching detail and master keys that
are specified by different range positions on the stage command argument. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
Change.htm: the
output of example #4 is incorrect. This will be corrected. |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
CHOP stage command:
is incorrectly defaulting to an ANYCASE setting, unless specified otherwise.
This will be corrected. |
|
Version
2.0 (test version) 04 February 2012 |
Pipelines 2.0
for ooRexx 32bit version 4+ has the following changes:
|
● |
Pipelines has been re-designed to work under/support the ooRexx
environment. This is a significant development and chapter in the evolution
of Pipelines. This re-design has involved extensive re-structuring of the
StageManager and the addition of new stages and API functions to support user
stages that might make use of access to ooRexx variables, stems and arrays. All further versions of Pipelines will be for use
under ooRexx. |
||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||
|
● |
The Regular Expression
Component Library that was employed by Pipelines (supplied by Tropic Software
East) has been replaced by the TR1 regex
implementation supplied by Microsoft. This change does not affect the syntax
of any of the stage commands that support regular expressions, however; the
following stage command DLL's have been modified to support the TR1
implementation:
This change also
means that the main online Pipelines and stage documentation, and executable
examples along with Pipelines Stage DLL Wizard; have been updated to reflect
the new regular expression interface. |
||||||||||||||||||||||||
|
● |
Under certain conditions;
when a required stage output stream connection is missing, the application abend's with a protection exception. This has been
corrected. |
||||||||||||||||||||||||
|
● |
FILEOUT (>)
stage command: is issuing canned message #2071 when it discovers that its
secondary input stream (and the stage expects to read a secondary input
stream record which specifies the output filename) is not connected. This may
happen in a pipeline which processes (for example; the output from the FLIST
stage command – when no files are found) and as a result, does not produce
any input to the FILEOUT (>) stage. This has been corrected. |
|
Version
1.9 25 May 2011 |
Pipelines version 1.9 has
the following changes:
|
● |
STRIP
stage command: the addition of a maxstrip
operand; which specifies the
maximum number of occurrences of the target character(s) or string to remove. |
|
● |
DELAY
stage command: did not write the argument portion of an input record to it's output stream when a
millisecond delay value is specified. This has been corrected. |
|
● |
A new TRACE option keyword; which will allow the step by step inspection of
input and output stream records. In order to support this; a shared file
segment has been implemented; to allow IPC notifications between pipelines
that are attached to the same console. This will allow the tracing of called
pipelines to be handled in the caller's console. Canned messages #1030,
#1031, #1032 & #1033 have been added to support the new TRACE
functionality. |
|
● |
The 'Home' hyperlink in the (target install
directory) '..\Examples\Readme.htm' document is
incorrectly pointing to the Pipelines version 1.7 online documentation page.
This has been corrected. |
|
● |
The substitution placeholder &VERSION
incorrectly returns a version number of 1.7. This has been corrected. |
|
● |
LITERAL stage command: an error was introduced
when restructuring the code for version 1.8. This means that the example:
ASCII32-126.ppl terminates with canned message #43. This has been corrected. |
|
● |
LOOKUP stage command: caused an abend issuing canned message
#41; when it determines that an input master record is out of sequence. This
has been corrected. |
|
● |
RUNPIPE stage
command: the addition of a QUIET sub-option on the NOWAIT operand; which will
launch the target pipeline (without an attached console) as a stand-alone
background pipeline. |
|
● |
FILEIN
(<) stage command: when FILEIN (<) was specified in position where it
is not the first stage in the pipeline and the file does not exist or cannot
be opened due to an error, the FILEIN (<) stage now issues a warning
message (instead of an error message) and continues to process its input
records. |
|
● |
SHELLEXECUTE stage command:
the addition of a MAXRC synonym. |
|
● |
CONSOLE stage command:
was incorrectly displaying certain characters when writing output records to
the console; this has been corrected. It has also been amended to support the
TRACE ON option. |
|
● |
CALLPIPE,
IN and OUT stage commands: have been amended to support the TRACE ON option. |
|
● |
A
fix for the CApplMessage class. CApplMessage was not rendering extended ASCII
characters correctly. This has been corrected. |
|
● |
The
option keywords MONITOR and PRIORITY were not restricted to a single
definition per pipelines instance (ie; in a
pipelines file) as described in the documentation. This has been corrected. |
|
● |
The
examples folder 'Handle arguments' in the online help directory structure and
the download package directory structure have been renamed to 'Subroutine'. |
|
Version 1.8 02
September 2010 |
Pipelines version 1.8 has the following
changes:
|
● |
SPECS stage command: the NEXTWORD operand will no longer insert a
SPACE into the output record, before appending data; if that data is null. |
|
● |
A new DELAY stage command: which waits until a
particular time of day or until a specified interval of time has elapsed
before copying a primary input stream record to its primary output stream.
Canned messages #64 and #65 have been added to support this new stage
command. |
The following two fixes are important; they rectify serious errors in
previous versions of Pipelines.
|
● |
SHELLEXECUTE stage command: was not capturing all of the executed
process' STDOUT and STDERR output data. This has been corrected. The stage command
has also been redesigned; the new design includes the BATCH operand; which
specifies that should
an input record argument contain a syntax error or invalid data, or that the
actual command fails to execute; SHELLEXECUTE will issue the appropriate error
message, but it will continue to read, process and execute input record
commands until it reaches end-of-file on its primary input stream.
SHELLEXECUTE will only support the BATCH operand argument; all other
arguments will be defined by each primary input stream record. Canned message
#66 has been added to support this change in functionality. |
|
● |
CALLPIPE stage command: was not capturing all of the called
pipeline's primary and secondary output data. This has been corrected. |
|
Version 1.7 09 May
2010 |
Pipelines version 1.7 has the following
changes:
|
● |
REVERSE stage command: has been removed. You will now have to use the REVERSE operand of the SPECS stage command to reverse record data. SPECS allows
you to reverse the data in a range of word, field and column output
specifications, whereas the REVERSE stage was only able to reverse the
contents of an entire record. The following example shows how you might use the SPECS stage command
to reverse the contents of columns 10 through 20 of an input record: 'pipe ...', '| specs .. 10-20 reverse ..', '| ...'
|
|
● |
SHELLEXECUTE stage command: did not actually support
the WAIT and NOWAIT operands (contrary to the documentation for version 1.6).
These have now been added. In addition; Pipelines 1.6 issues an incorrect
canned message #63, it currently reads: 'You cannot specify the WAIT operand in combination with the CMD
operand.' It has now been corrected, and reads: 'You cannot specify the NOWAIT operand in combination with the CMD
operand.' Documentation file: Messages.htm (which was missing any reference to
canned message #63) now includes the correct message. |
|
● |
LOCATE and NLOCATE stage
commands: support for the numeric comparison operators: <, <=, ==, !=, >= and >. |
|
● |
FILELIST stage command: support for the alternate
stage command name: LISTFILE. |
|
● |
A new DEAL stage
command: which writes primary input stream records to its connected output
streams in either sequential order; starting with the primary output stream,
or some other order specified on its secondary input stream. |
|
Version 1.6 30
June 2009 |
Pipelines version 1.6 has the following
changes:
|
● |
This version of Pipelines is currently only available for the .NET
2.0 platform. |
|
● |
A Pipelines Visual Studio/VC++ Stage Command API
mechanism (phase 1: VS8 - VC++ 9). The API provides all the initialisation and runtime routines that
support the current builtin stage set; comprising stage command parsing,
runtime extraction and stream connection routines and a stage DLL project
wizard which creates a fully functional skeletal Pipelines stage DLL. |
|
● |
In order to keep the Pipelines download package to a reasonable and
manageable size and at the same time provide me with the flexibility to
publish Pipelines features which may have already been developed but not yet
documented, as and when they are deemed fit for use; the Pipelines
documentation is now available online and no longer provided with the
download package. |
|
● |
FILELIST stage command: the file date and time
values have been changed to represent 'local-time'; this brings the field's
in line with the DOS DIR and Windows file date and time property values. |
|
● |
The
default regular expression mode was incorrectly defined as 1; this has been
changed to a value of 2 (the standard Perl regular expression parsing mode). |
|
● |
A
re-design and extension of the SNAKE stage
command. SNAKE now only delays those records which comprise a complete matrix, that
is to say, if you specify the numrows operand; once the stage has read
enough input records to build an output matrix of numcols by numrows;
the output matrix is written to the primary output stream, the allocations
for that matrix are released and the stage begins building the next matrix.
This re-design improves the performance of the stage by a significant factor
(for multiple matrices). SNAKE
now supports a PAD operand and pad character operand which allows the
specification of the character which is used to extend shorter input records
for output alignment. |
|
● |
An
increase in record throughput; the streamline of the StageManager i/o request
mechanism provides much needed improvement in most pipeline constructions. |
|
● |
A
new MONITOR option keyword; which allows you to turn off stall detection. |
|
● |
LITERAL stage command: the addition of a PREFACE and APPEND operand; which
specify the order in which the command argument and primary input stream
records are processed. |
|
● |
A
new SHELLEXECUTE stage command: which
allows you to specify a (system) shell command, or indeed any other
executable process, either synchronously or asynchronously. This
addition of the SHELL stage command allows you to construct pipelines which
perform housekeeping; file and directory operations; for example; deleting,
renaming and attribute modification. |
|
● |
Ill-placed
and unnecessary registry definitions have been removed; this enables the RUNPIPE stage command to operate much more efficiently. |
|
● |
The
correction of some broken hyperlinks in the Filein.htm document. |
|
Version 1.5 22
April 2008 |
Pipelines version 1.5 has the following
changes:
|
● |
Pipelines version 1.5 is the last version of Pipelines to target the
.NET 1.0 platform; all future versions of Pipelines will target .NET 2.0 as a
minimum platform requirement. |
|
● |
BUFFER stage command: a REVERSE operand; which specifies that the
contents of the buffer are written to the primary output stream in reverse
record order. |
|
● |
FILEIN
(<) stage command: a REVERSE operand; which specifies that an input file
is read in reverse record order. The
REVERSE process uses a bespoke read process which traverses an input file in
reverse order. To ensure that the FILEIN (<), FILEIN (<) REVERSE and IN
stage commands all produce identical output records (so that multiple,
multi-stream pipelines are consistent in their handling of the records) all
three processes now split their input at New-Line characters only; individual
and groups of Carriage-Return characters are discarded. |
|
● |
CONSOLE
stage command: determines if its primary output stream is connected when the
stage is dispatched; if true, once CONSOLE determines that its primary output
stream has become disconnected, the stage ends. |
|
Version 1.4 22
March 2008 |
Pipelines version 1.4 has the following
changes:
|
● |
Per-pipeline
support for a range of regular expression flavours. |
||||||||||||||||||||||||
|
● |
Implementation
of a regular expression option for the following stage commands:
The PATTERN, ANYOF and charrange options remain, where appropriate;
as they provide the fastest and most efficient method of searching for
expressions which can be described by a range of characters or a simple
pattern mask. |
||||||||||||||||||||||||
|
● |
Per-stage
command history of change details. |
||||||||||||||||||||||||
|
● |
A
new SNAKE stage command: which builds multi-column
matrices. |
||||||||||||||||||||||||
|
● |
CHANGE
stage command: was not writing unchanged records to its secondary output
stream; this has been corrected. |
||||||||||||||||||||||||
|
● |
JOIN stage command: no longer
discards null input records; JOIN concatenates consecutive
input records, up to, but not including a null input record. |
||||||||||||||||||||||||
|
● |
Other
miscellaneous changes and corrections. |
|
Version 1.3 04
November 2007 |
Pipelines version 1.3 has the following
changes:
|
● |
NLOCATE stage command: selected and unselected records were not being
written to the correct output streams; this has been corrected. |
|
● |
CALLPIPE and OUT stage commands: have been
extended: Pipelines now allows you to call sub-routine pipelines which route
selected and unselected output records back to its caller; where OUT stream 0
corresponds to the primary output stream of the CALLPIPE stage of the calling
pipeline, and OUT stream 1 corresponds to the secondary output stream of the
CALLPIPE stage of the calling pipeline. This enhancement is significant; it
makes possible the construction of filter pipelines that operate in a similar
fashion to many of the builtin stage commands. |
|
● |
Other
miscellaneous changes and corrections. |
|
Version 1.2 09
October 2007 |
Pipelines version 1.2 has the following
changes:
|
● |
FILEIN (<) stage command: changed to deny write access to open
input files. This is complementary to the FILEOUT (>) and
FILEAPPEND (>>) stage commands; which deny read access to open output
files. |
|
● |
Documentation files: Zone.htm and Casei.htm no longer
specify example pipelines which use the LOCATE stage command. LOCATE does not
use the pre-process functionality any more; it
now provides full multi-word-field-column selection and a non-case-sensitive
selection comparison. |
|
● |
SPECS
stage command: the operands STRIP, alignment LEFT, CENTER and RIGHT, now
only remove spaces (x'20') and tabs (x'09') before positioning the data in
the output record. |
|
Version 1.1 21
September 2007 |
Pipelines version 1.1 has the following
changes:
|
● |
This document: History of change.htm has been moved from the
install package to the tenfiftytwo.co.uk/pipelines website. |
|
● |
Example pipelines: Machine snapshot.ppl and MachineDiff.ppl
have been changed. They both now correctly write to and read from the users' My
Documents folder. |
|
Version 1.0 06
September 2007 |
Pipelines
version 1.0 comprises the following features:
|
● |
Multi-pipeline
processing |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
Multi-stream
connectivity |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
Pipeline
stall-detection and dump mechanism |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
Priority
dispatch control |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
Argument
and place-holder substitution |
||||||||||||||||||||||||||||||||||||||||||||
|
● |
The following bultin stage
commands:
|
|
|