This chapter describes how extensions to Regina are implemented. The whole contents of this chapter is specific for Regina.

  1. Why Have Extensions

Why do we need extensions? Well, there are a number of reasons, although not all of these are very good reasons:




Extensions arise from holes in the functionality. Whether they will survive or not depends on how they are perceived by programmers; if perceived as useful, they will probably be used and thus supported in more interpreters.

  1. Extensions and Standard REXX

In standard REXX, the OPTIONS instruction provides a "hook" for extensions. It takes any type of parameters, and interprets them in a system-dependent manner.


The format and legal values of the parameters for the OPTIONS instruction is clearly implementation dependent [TRL2, p62].

  1. Specifying Extensions in Regina

In Regina there are three level of extensions. Each independent extension has its own name. Exactly what an independent extension is, will depend on the viewer, but a classification has been done, and is listed at the end of this chapter.


At the lowest level are these "atomic" extensions. Then there are some "meta-extensions". These are collections of other extensions which belongs together in some manner. If you need the extension for creating "buffers" on the stack, it would be logical to use the extension to remove buffers from the stack too. Therefore, all the individual extensions for operations that handle buffers in the stack can be named by such a "meta-extensions". At the end of this chapter, there is a list of all the meta-extensions, and which extensions they include.


At the top is "standards". These are sets of extensions that makes the interpreter behave in a fashion compatible with some standard. Note that "standard" is used very liberally, since it may refer to other implementations of REXX. However, this description of how the extensions are structured is only followed to some extent. Where practical, the structure has been deviated.

  1. The Trouble Begins

There is one very big problem with extensions. If you want to be able to turn them on and off during execution, then your program has to be a bit careful.


More and more REXX interpreters (including Regina seem to do a parsing when the interpreter is started. The "old" way was to postpone the parsing of each clause until it was actually executed. This leads to the problem mentioned.


Suppose you want to use an extension that allows a slightly different syntax, for the sake of the argument, let us assume that you allow an expression after the SELECT keyword. Also assume that this extension is only allowed in extended more, not in "standard mode". However, since Regina parses the source code only once (typically at the starts of the program), the problem is a catch-22: the extension can only be turned on after parsing the program, but it is needed before parsing. This also applies to a lot of other REXX interpreters, and all REXX compilers and preprocessors.


If the extension is not turned on during parsing, it will generate a syntax error, but the parsing is all done before the first clause is executed. Consequently, this extension can not be turned on during execution, it has to be set before the parsing starts.


Therefore, there are two alternative ways to invoke a set of extensions; neither of which is implemented in Regina.


  1. The Format of the OPTIONS clause

The format of the OPTIONS clause is very simple, it is followed by any REXX string expression, which is interpreted as a set of space separated words. The words are treated strictly in order from left to right, and each word can change zero or more extension settings.


Each extension has a name. If the word being treated matches that name, that extension will be turned on. However, if the word being treated matches the name of an extension but has the prefix NO, then that extension is turned off. If the word does not match any extensions, then it is simply ignored, without creating any errors or raising any conditions.


Example: Extensions changing parsing


An example of this is the LINES BIF. In the following piece of code the same BIF returns different data:


/* file 'aa' contains 5 lines */

options FAST_LINES_BIF_DEFAULT

do i=1 to 2

if i=2 then OPTIONS NOFAST_LINES_BIF_DEFAULT

say lines('aa')

end


In the first iteration of the loop, LINES('aa') returns 1, indicating that there is at least 1 line remaining the the stream 'aa'. However, in the second iteration of the loop, LINES('aa') will return 5, indicating that there are 5 lines remaining in the stream.


Regina's frequent usage of extensions may slow down execution. To illustrate how this can happen, consider the OPEN() extra built-in function. As this is an extension, it might be dynamically included and excluded from the scope of currently defined function. Thus, if the function is used in a loop, it might be in the scope during the first iteration, but not the second. Thus, Regina can not cache anything relating to this function, since the cached information may be outdated later. As a consequence, Regina must look up the function in the table of functions for each invocation. To avoid this, you can set the extension CACHEEXT, which tells Regina to cache info whenever possible, without regards to whether this may render useless later executions of OPTIONS.

  1. The Fundamental Extensions

Here is a description of all "atomic" extensions in Regina:


[AREXX_BIFS]

This option allows the user to enable or disable the AREXX BIFs introduced into Regina 3.1. The default is AREXX_BIFS on Amiga and AROS, but NOAREXX_BIFS on all other platforms.

[AREXX_SEMANTICS]

With the introduction of AREXX BIFs into Regina 3.1, differences in the semantics of a number of BIFs resulted. These BIFs that differ between Standard Regina and AREXX are OPEN(), CLOSE() and EOF(). This OPTION specifies that the AREXX semantics be used for these BIFs. The default is to use Regina semantics for these BIFs.

[BUFTYPE_BIF]

Allows calling the built-in function BUFTYPE(), which will write out all the contents of the stack, indicating the buffers, if there are any. The idea is taken from VM/CMS, and its command named BUFTYPE.

[CACHEEXT]

Tells Regina that information should be cached whenever possible, even when this will render future execution of the OPTIONS instruction useless. Thus, if you use e.g. the OPEN() extra built-in function, and you set CACHEEXT, then you may experience that the OPEN() function does not disappear from the current scope when you set the NOOPEN_BIF extension.


Whether or not a removal of an extension really do happen is unspecified when CACHEEXT has been called at least once. Effectively, info cached during the period when CACHEEXT was in effect might not be "uncached". The advantage of CACHEEXT is efficiency when you do not need to do a lot of toggling of some extension.

[DESBUF_BIF]

Allows calling the built-in function DESBUF(), to remove all contents and all buffers from the stack. This function is an idea taken from the program by the same name under VM/CMS.

[DROPBUF_BIF]

Allows calling the built-in function DROPBUF(), to removed one of more buffers from the stack. This function is an idea take from the program by the same name under VM/CMS.

[EXT_COMMANDS_AS_FUNCS]

When Regina resolves an expression to a function, and that function is not a built-in or a registered external

function, Regina attempts to execute the function as an operating system command. With

NOEXT_COMMANDS_AS_FUNCS set, Regina will return error 43; "Routine not found".

EXT_COMMANDS_AS_FUNCS is the default.

[FAST_LINES_BIF_DEFAULT]

The LINES() BIF in versions of Regina prior to 0.08g returned the actual number of lines available in a stream. Since then, the LINES() BIF has been changed to only return 0 or 1. This was done for two reasons. First, it is faster, and secondly. the ANSI standard allows for an option to return the actual number of lines. This OPTION is for backwards compatibility with programs written assuming the prior behaviour of the LINES() BIF. FAST_LINES_BIF_DEFAULT is the default.

[FLUSHSTACK]

Tells the interpreter that whenever a command clause instructs the interpreter to flush the commands output on the stack, and simultaneously take the input from the stack, then the interpreter will not buffer the output but flush it to the real stack before the command has terminated. That way, the command may read its own output. The default setting for Regina is not to flush, i.e. NOFLUSHSTACK, which tells interpreter to temporary buffer all output lines, and flush them to the stack when the command has finished.

[INTERNAL_QUEUES]

Regina implements multiple named queues both as part of the interpreter, and as an external resource. The use of the RXQUEUE() BIF, will make Regina use the external queueing mechanism. This OPTION allows the exclusive use of Regina's internal queueing mechanism. NOINTERNAL_QUEUES is the default.

[LINEOUTTRUNC]

This options tells the interpreter that whenever the LINEOUT() built-in function is executed for a persistent file, the file will be truncated after the newly written line, if necessary. This is the default setting of Regina, unless your system does not have the ftruncate() system call. The complement option is NOLINEOUTTRUNC.

[MAKEBUF_BIF]

Allows calling the built-in function MAKEBUF(), to create a buffer on the stack. This function is an idea taken from a program by the same name under VM/CMS.

[PRUNE_TRACE]

Makes deeply nested routines be displayed at one line. Instead of indenting the trace output at a very long line (possibly wrapping over several lines on the screen). It displays [...] at the start of the line, indicating that parts of the white space of the line has been removed.

[REGINA_BIFS]

This OPTION allows the user to turn off any non-ANSI extenion BIFs. The default is REGINA_BIFS.

[STDOUT_FOR_STDERR]

All output that Regina would normally write to stderr, such as TRACE output and errors, are written to stdout instead. This is useful if you need to capture TRACE output and normal output from SAY to a file in the order in which the lines were generated. The default is NOSTDOUT_FOR_STDERR.

[STRICT_ANSI]

This OPTION results in interpretation of a program to strict ANSI standards, and will reject any Regina extensions. NOSTRICT_ANSI is the default.

[STRICT_WHITE_SPACE_COMPARISON]

This OPTION specifies if ANSI rules for non-strict comparisons are applied. Under ANSI, when doing non-strict comparisons, only the space character is stripped from the two comparitors. Under Regina's default behaviour, all whitespace characters are stripped. NOSTRICT_WHITE_SPACE_COMPARISON is the default.

[TRACE_HTML]

This OPTION generates HTML <PRE> and </PRE> tags around TRACE output, to enable tracing from

within CGI scripts. The default is NOTRACE_HTML.


Note: OPEN_BIF, FIND_BIF, CLOSE_BIF and FILEIO OPTIONs have been removed in Regina 3.1

  1. Meta-extensions

[BUFFERS]

Combination of BUFTYPE_BIF, DESBUF_BIF, DROPBUF_BIF and MAKEBUF_BIF.

  1. Semi-standards

[CMS]

A set of extensions that stems from the VM/CMS operating system. Basically, this includes the most common extensions in the VM/CMS version of REXX, in addition of some functions that perform task normally done with commands under VM/CMS.

[VMS]

A set of interface functions to the VMS operating system. Basically, this makes the REXX programming under VMS as powerful as programming directly in DCL.

[UNIX]

A set of interface functionality to the Unix operating system. Basically, this includes some functions that are normally called as commands when programming Unix shell scripts. Although it is possible to call these as commands in Regina, there are considerable speed improvements in implementing them as built-in functions.

  1. Standards

The folowing table shows which options are available in different REXX Langugae Levels, and the default settings applicable for Regina.


[ANSI]

REXX Language level 5.0, as described in [ANSI]. This can be set wth the STRICT_ANSI OPTION in Regina.

[REGINA]

REXX Language level 5.0, plus extensions, as implemented by Regina 3.1 and above.

[SAA]

REXX Language level ??, as defined by IBM's System Application Architecture [SAA].

[TRL1]

REXX Language level 3.50, as described in [TRL1].

[TRL2]

REXX Language level 4.00, as described in [TRL2].


Option

ANSI

REGINA

SAA

TRL1

TRL2

AREXX_BIFS

no

yes

no

no

no

AREXX_SEMANTICS

no

no

no

no

no

BUFTYPE_BIF

no

yes

??

no

no

CACHEEXT

no

no

no

no

no

DESBUF_BIF

no

yes

??

no

no

DROPBUF_BIF

no

yes

??

no

no

EXT_COMMANDS_AS_FUNCS

no

yes

??

no

no

FAST_LINES_BIF_DEFAULT

no

yes

??

no

no

FLUSHSTACK

no

no

??

no

no

INTERNAL_QUEUES

no

no

??

no

no

LINEOUTTRUNC

no

yes

??

no

no

MAKEBUF_BIF

no

yes

??

no

no

PRUNE_TRACE

no

yes

no

no

no

REGINA_BIFS

no

yes

no

no

no

STDOUT_FOR_STDERR

no

no

no

no

no

STRICT_ANSI

no

no

no

no

no

STRICT_WHITE_SPACE_COMPARISON

no

no

no

no

no

TRACE_HTML

no

no

no

no

no


Table of Contents

Extensions 1

1Why Have Extensions 1

2Extensions and Standard REXX 1

3Specifying Extensions in Regina 1

4The Trouble Begins 2

5The Format of the OPTIONS clause 2

6The Fundamental Extensions 3

7Meta-extensions 5

8Semi-standards 5

9Standards 6


7