A Rexx interface to the cURL library

Version 1.0

4 July 2001

1. Introduction
2. Overview
3. Functions: CURLINIT; CURLCLEANUP; CURLSETOPT; CURLPERFORM; CURLGETINFO; CURLVARIABLE; CURLLOADFUNC; CURLDROPFUNC
4. Errors
5. Using Rexx/CURL
6. History of Rexx/CURL

1. Introduction

This document describes the interface to the cURL package. cURL is a general purpose package that allows access to any URL-addressable resource. With Rexx/CURL you can access resources such as web pages, ftp sites, and telnet sessions under control of your Rexx program. Rexx/CURL is actually built on top of libcurl but for the purposes of this document it will be refered to simply as cURL.

2. Overview

Rexx/CURL consists of Rexx external functions that allows a Rexx program to access any URL. The basic concept of Rexx/CURL (and cURL), is that you specify a URL and all options appropriate to that URL, and then perform the URL access.
See Using Rexx/CURL for more details.

The Rexx/CURL external functions are:

CURLINIT - initialise the cURL interface
CURLCLEANUP - cleanup the cURL interface
CURLSETOPT - defines all the options appropriate to the type of URL you wish to access
CURLPERFORM - carries out the actions defined by the various calls to CURLSETOPT
CURLGETINFO - return information about the action carried out by CURLPERFORM
CURLVARIABLE - set or retrieve default run-time values

CURLLOADFUNCS - load all Rexx/CURL external functions
CURLDROPFUNCS - unload all Rexx/CURL external functions

3. Functions

This section provides the full syntax and usage of each function that comprises Rexx/CURL.

CURLINIT()

Initialises the cURL interface. The return value from this call is used as the first argument to most of the other Rexx/CURL functions.

Arguments:

none

Returns:

success:: any non-blank value
failure:: blank

CURLSETOPT(handle, option,option value)

This function is called to define all the parameters and data required to carry out the particular request. The option argument is case-insensitive. The table below defining the options available contains the equivalent cURL option value used in the C/C++ interface. Not all of the Rexx/CURL option strings are the same as the C/C++ equivalents; I've tried to be more consistent and clearer with the name of the option. The C/C++ equivalents are there if you wish to read alternate definitions of these options. See the curl_easy_setopt() function documentation.

Arguments:

handle

The value returned from CURLINIT.

option

This is the string identifying the option to set.

Rexx/CURL Option	Description	cURL Equivalent Option
CLOSEPOLICY	Specify the type of disconnection policy to use if the connection cache is filled. This is only applicable if you potentially use more than 5 concurrent connections. Can be one of OLDEST, or LEAST_RECENTLY_USED.	CURLOPT_CLOSEPOLICY
CONNECTTIMEOUT	To limit the time it takes to connect to the server, set this value to the number of seconds.	CURLOPT_CONNECTTIMEOUT
COOKIE	If you want to pass a cookie to the server, set this option to the cookie. The format of the cookie is: name=contents, where name is the name of the cookie to be set.	CURLOPT_COOKIE
COOKIEFILE	The name of a file from which cookie data is read for the performed action. The cookie file can contain Netscape/Mozilla formated cookies, or regular HTTP header format. This file must exist.	CURLOPT_COOKIEFILE
CUSTOMREQUEST	To carry out an HTTP request command other than GET or HEAD, pass the command in this option.	CURLOPT_CUSTOMREQUEST
EGDSOCKET	The name of the Entropy Gathering Socket which is used to seed the SSL random engine.	CURLOPT_EGDSOCKET
ERRFILE	The name of a file into which any error output from the performed action is written. If this file exists, it will be over-written	CURLOPT_STDERR
FAILONERROR	Set this option to a 1 or Y to get cURL to fail, rather than return the page, if the HTTP return code is greater than or equal to 300.	CURLOPT_FAILONERROR
FILETIME	Set this option to a 1 or Y to indicate that cURL should attempt to retrieve the modification date of the remote document. Not all servers will respond to this type of request. To retrieve the date, call CURLGETINFO with the FILE_TIME option.	CURLOPT_FILETIME
FOLLOWLOCATION	Set this option to a 1 or Y to get cURL to follow any Location: headers in the specified site.	CURLOPT_FOLLOWLOCATION
FORBIDREUSE	TBD	CURLOPT_FORBID_REUSE
FRESHCONNECT	Set this option to a 1 or Y to indicate that the next transfer should use a new connection.	CURLOPT_FRESH_CONNECT
FTPAPPEND	Set this option to a 1 or Y to indicate that the remote file is to be appended to rather than replaced when an FTP upload is to be carried out.	CURLOPT_FTPAPPEND
FTPCMDSAFTER	A list of FTP commands to be executed after the URL is accessed. The list of commands is specified as a stem name; ie the supplied string must end in a period, and represent a valid Rexx array.	CURLOPT_POSTQUOTE
FTPCMDSBEFORE	A list of FTP commands to be executed before the URL is accessed. The list of commands is specified as a stem name; ie the supplied string must end in a period, and represent a valid Rexx array.	CURLOPT_QUOTE
FTPCRLF	Set this option to a 1 or Y to indicate that LF charcaters should be converted to CRLF on FTP uploads.	CURLOPT_CRLF
FTPLISTONLY	Set this option to a 1 or Y to indicate that only filenames are to be returned when the URL option specifies an FTP directory. Normally, file sizes, dates etc. are returned.	CURLOPT_FTPLISTONLY
FTPPORT	Set this option to a string to use as the parameter to the FTP PORT command. The parameter can be an IP address, a hostname, an interface name (undex Unix), or '-' to use the client machine's default IP address.	CURLOPT_FTPPORT
HEADER	Set this option to a 1 or Y to get cURL to return header information for those protocols, like HTTP, that have seperate headers and footers.	CURLOPT_HEADER
HEADERFILE	The name of a file into which headers from the performed action are written. If this file exists, it will be over-written	CURLOPT_WRITEHEADER
HEADERSTEM	Specify the stem variable of an array into which any headers from the performed action are written. If the array exists, it will be over-written. Where possible, each entry in the array will contain a single line.	CURLOPT_WRITEHEADER
HTTPHEADER	To pass a series of HTTP headers to the server, set this option to a valid Rexx stem. Any headers included in this option, that would have normally been generated internally by cURL, will be replaced.	CURLOPT_HTTPHEADER
HTTPPOST	Set this option to a 1 or Y to indicate that a regular (application/x-www-form-urlencoded) HTTP POST is to be carried out. Most HTTP forms are of this type. See POSTFIELDS for details on how to specify the content of the form fields.	CURLOPT_POST
HTTPPOSTDATA	Setting this option indicates that you wish to issue a multipart/formdata HTTP POST. You pass the data that is posted as a valid Rexx stem.	CURLOPT_HTTPPOST
HTTPPOSTFIELDS	Specify the content of the fields to be filled in with a HTTPPOST. The passed parameter is a Rexx array, with each item in the array a name/value pair. eg field.1 = 'email=M.Hessling@qut.edu.au'	CURLOPT_POSTFIELDS CURLOPT_POSTFIELDSIZE
HTTPPROXYTUNNEL	Set this option to a 1 or Y to tunnel all non-http operations through the HTTP proxy.	CURLOPT_HTTPPROXYTUNNEL
HTTPPUT	Set this option to a 1 or Y to get indicate that a HTTP PUT command is issued for the URL. The file to be uploaded must be specified with INFILE.	CURLOPT_PUT
INFILE	The name of a file from which data is read for the performed action. This file must exist. For ftp uploads, this is the file to upload.	CURLOPT_INFILE CURLOPT_INFILESIZE
INTERFACE	To specify an outgoing interface other than the default, pass the interface in this option. The interface can be specified as an IP address, a hostname, or an interface name (undex Unix).	CURLOPT_INTERFACE
KRB4LEVEL	TBD	CURLOPT_KRB4LEVEL
LOWSPEEDLIMIT	This option should contain a number representing the bytes per second that cURL will use as the lowest transfer rate to run at before it aborts the session as being too slow.	CURLOPT_LOW_SPEED_LIMIT
LOWSPEEDTIME	This option should contain a number representing the number of seconds that cURL will use as the lowest transfer time to run for before it aborts the session as being too slow.	CURLOPT_LOW_SPEED_TIME
MAXCONNECTS	You can specify how many persistent connections cURL is to maintain. The default is 5 and unless you know what you are doing leave it alone.	CURLOPT_MAXCONNECTS
MAXREDIRS	To limit the number of redirections followed, set this value with this option. This option only makes sense when FOLLOWLOCATION is also set.	CURLOPT_MAXREDIRS
NETRC	Set this option to a 1 or Y to get cURL to read your ~/.netrc for the username and password for the remote site.	CURLOPT_NETRC
NOBODY	Set this option to a 1 or Y to get cURL to not return the body information for those protocols, like HTTP, that have seperate headers and footers.	CURLOPT_NOBODY
OUTFILE	The name of a file into which any output from the performed action is written. If this file exists, it will be over-written.	CURLOPT_FILE
OUTSTEM	Specify the stem variable of an array into which any output from the performed action is written. If the array exists, it will be over-written. Where possible, each entry in the array will contain a single line.	CURLOPT_FILE
PROXY	If you need to use a HTTP proxy to access the outside world, specify it with this option. The format of the proxy string is host[:port]. The default port number is 1080.	CURLOPT_PROXY
PROXYUSERPWD	Specify the username/password to use for the the HTTP proxy connection. The format is username[:password]. If the password is omitted, you will be prompted for it.	CURLOPT_PROXYUSERPWD
RANDOMFILE	The name of a file from which is used to seed the SSL random engine. This file must exist.	CURLOPT_RANDOM_FILE
RANGE	Specify the required range you want in the format X-Y. I have no idea what this is!	CURLOPT_RANGE
REFERER	Set this option to a string to set the header field: referer: in the HTTP request.	CURLOPT_REFERER
RESUMEFROM	Set this option to the byte at which you want a request to start from. This is particularly useful for restarting an FTP download that was interrupted.	CURLOPT_RESUME_FROM
SSLCERT	To set a SSL certificate, set this option to the filename containing the certificate. The certificate should be in PEM format.	CURLOPT_SSLCERT
SSLCERTPASSWD	The password associated with the SSL certificate set by SSLCERT can be set with this option. If you don't supply the password with this option, you will be prompted for it.	CURLOPT_SSLCERTPASSWD
SSLPEERCERT	The name of a file from which the SLL peer certificate data is read for the performed action. This file must exist.	CURLOPT_CAINFO
SSLVERIFYPEER	Set this option to a 1 or Y to indicate that the SSL peer's certificate should be verified. The SSL peer certificate must be specified with the SSLPEERCERT option.	CURLOPT_SSL_VERIFYPEER
SSLVERSION	To over-ride the default SSL version used, pass this option in as a number.	CURLOPT_SSLVERSION
TIMECONDITION	TBD	CURLOPT_TIMECONDITION
TIMEOUT	Specify the number of seconds in which the whole transaction is to complete. Note that this timeout period also includes the time taken to do name lookups, so don't specify too short a period.	CURLOPT_TIMEOUT
TIMEVALUE	TBD	CURLOPT_TIMEVALUE
TRANSFERTEXT	Set this option to a 1 or Y to get indicate that FTP transfers are to be done in ASCII mode. For LDAP requests, the details are returned in plain text rather than HTML.	CURLOPT_TRANSFERTEXT
UPLOAD	Set this option to a 1 or Y to indicate that an upload is to be done, rather than a download.	CURLOPT_UPLOAD
USERAGENT	Set this option to a string to set the header field: user-agent: in the HTTP request. This can be useful to imitate different browser clients.	CURLOPT_USERAGENT
USERPWD	Specify the username/password to use for this connection. The format is username[:password]. If the password is omitted, you will be prompted for it.	CURLOPT_USERPWD
URL	The URL against which the access is to be made.	CURLOPT_URL
VERBOSE	Set this option to a 1 or Y to get cURL to display lots of details about what it is doing.	CURLOPT_VERBOSE

option value

The value of the option to set.

Returns:

success:: blank
failure:: blank; On failure CURLERROR.INTCODE is set to a non-zero value. If this value is 1 (one) then CURLERROR.CURLCODE is also set to a non-zero value.

CURLPERFORM(handle)

Once all the options have been set up, call this function to carry out the transfer.

Arguments:

handle: The value returned from CURLINIT.

Returns:

success:: blank
failure:: blank; On failure CURLERROR.INTCODE is set to a non-zero value. If this value is 1 (one) then CURLERROR.CURLCODE is also set to a non-zero value.

CURLGETINFO(handle, option)

Sets the default database connections to be that which is specified or if no connection name is specified, the current connection name is returned. Arguments:

handle

The value returned from CURLINIT.

option

This is the string identifying the information to retrieve.

Rexx/CURL Option	Description	cURL Equivalent Option
CONNECT_TIME	Returns the number of seconds it took to connect to the remote server.	CURLINFO_CONNECT_TIME
CONTENT_LENGTH_DOWNLOAD	Returns the length of the contents returned. This is the value returned by the header field; Content-Length:	CURLINFO_CONTENT_LENGTH_DOWNLOAD
CONTENT_LENGTH_UPLOAD	Returns the length of the specified upload size.	CURLINFO_CONTENT_LENGTH_UPLOAD
EFFECTIVE_URL	Returns the last used effective URL.	CURLINFO_EFFECTIVE_URL
FILE_TIME	The time of the remote document received. This is only returned if setopt FILETIME is called. WHT FORMAT? CONVERT TIME_T TO TIME('N')	CURLINFO_FILETIME
HEADER_SIZE	Returns the length of all headers returned.	CURLINFO_HEADER_SIZE
HTTP_CODE	Returns the last received HTTP code. Only useful if the last URL requested was HTTP.	CURLINFO_HTTP_CODE
NAMELOOKUP_TIME	Returns the time in seconds for the time taken to resolve the remote server name.	CURLINFO_NAMELOOKUP_TIME
PRETRANSFER_TIME	Returns the total time taken from the start up until the transfer is about to begin. This time includes all pre-transfer commands and negotiations.	CURLINFO_PRETRANSFER_TIME
REQUEST_SIZE	Returns the total size of all requests. Note that this may be for more than one request if FOLLOWLOCATION was set.	CURLINFO_REQUEST_SIZE
SIZE_DOWNLAOD	Returns the total number of bytes downloaded.	CURLINFO_SIZE_DOWNLOAD
SIZE_UPLOAD	Returns the total number of bytes uploaded.	CURLINFO_SIZE_UPLOAD
SPEED_DOWNLOAD	Returns the average download speed in bytes OR K? per second.	CURLINFO_SPEED_DOWNLOAD
SPEED_UPLOAD	Returns the average upload speed in bytes OR K? per second.	CURLINFO_SPEED_UPLOAD
SSL_VERIFY_RESULT	Returns the result of the SSL certificate verification requested by SSLVERIFYPEER.	CURLINFO_SSL_VERIFYRESULT
TOTAL_TIME	Returns the time in seconds for the complete transfer.	CURLINFO_TOTAL_TIME

Returns:

success:: blank
failure:: blank; On failure CURLERROR.INTCODE is set to a non-zero value. If this value is 1 (one) then CURLERROR.CURLCODE is also set to a non-zero value.

CURLVARIABLE(variable name[,variable value])

Set or get the value for the specified variable.

The following variables are available in all implementations:

VERSION (readonly) the version of Rexx/CURL, consisting of:
- package name - rexxcurl
- Rexx/CURL version - numerical version; eg. 1.0
- Rexx/CURL date - Rexx standard date format; eg. 4 Jul 2001
- OS platform - current operating system
- cURL version - version of cURL library: libcurl version
eg. REXXCURL 1.0 4 Jul 2001 UNIX libcurl 7.8

DEBUG (setable) level of debugging requested.
- 0 - no debugging information displayed (default)
- 1 - Rexx variables displayed as set
  Equivalent to -v command line flag.
- 2 - function entry/exit information displayed
  Equivalent to -d command line flag.
- 4 - internal tracing information displayed
  Equivalent to -D command line flags.
Any of the above values may be added together to combine their effects. eg. 6 is equivalent to -dD on the command line.
DEBUGFILE (setable) file where debug information is sent.
Any valid filename. If the file exists it will be overwritten If not specified, all dubugging information gets written to stderr.
ERROR (setable) prefix for error code variables.
any valid Rexx variable name; usually a stem name. The default is CURLERROR.

Arguments:

variable name
The name of the variable who's value is to be set or retrieved.
variable value: The value that the variable to be set to.

Returns:

with variable value specified:: success; blank if a valid variable name specified and it is able to be set;; failure; blank and CURLERROR.INTCODE set to a non-zero value.
with variable value NOT specified:: success; the current value of the variable; failure; blank and CURLERROR.INTCODE is set to a non-zero value.

CURLLOADFUNCS()

This function is used to load all the Rexx/CURL external functions. This function is called after the function has been loaded with the Rexx builtin function rxfuncadd().

Although this function is useful only for dynamic library implementations of Rexx/CURL, it can be called by the executable version of Rexx/CURL. In this case it does nothing.

Arguments:

none

Returns:

success:: zero
failure:: non-zero

CURLDROPFUNCS()

This function is used to terminate Rexx/CURL and free up all resources that have been used.

It should be called at the end of every Rexx/CURL program. In particular, this function should be called after a syntax error has been caught with SIGNAL ON SYNTAX.

Arguments:

none

Returns:

success:: zero
failure:: non-zero

4. Errors

The success or failure of each function call is determined by the Rexx compound variable; CURLERROR.INTCODE. If the function call succeeds, this will be set to zero. If the function call fails, this will be set to a non-zero value. If the value set is 1 (one), a cURL error occured, and CURLERROR.CURLCODE is set to the appropriate error code. Associated with both error code variables, is an equivalent textual error message. These are CURLERROR.INTERRM and CURLERROR.CURLERRM respectively.
The stem name initially set for the error variables is CURLERROR. (including trailing period). You can change this to any value you prefer, with a call to CURLVARIABLE with the ERROR argument.

Internal Errors:

    1  - Error from cURL
    2  - Invalid Number
    3  - Invalid Option
    4  - Out of memory
    5  - Invalid cURL handle
    6  - Invalid filename
    7  - Invalid boolean
    8  - Expecting a stem as parameter
    9  - Invalid variable name specified
   10  - Attempt to set a readonly variable name

5. Using Rexx/CURL

A typical Rexx/CURL program looks like the following program. This example will simply display the contents of my home page:

Call RXFuncAdd 'CURLLoadFuncs','rexxcurl','CURLLoadFuncs'
Call CURLLoadFuncs
curl = CURLInit()
if curl \= '' Then
Do
Call CURLSetOpt 'URL', "http://www.lightlink.com/hessling/"
If curlerror.intcode \= 0 Then Call Abort 'Error setting URL option'
Call CURLPerform curl
If curlerror.intcode \= 0 Then Call Abort 'Error performing action'
Call CURLCleanup curl
End
Call CURLDropFuncs()
Return 0
Abort:
Parse Arg msg
Say msg
If curlerror.intcode = 1 Then Say 'Internal error:' curlerror.intcode curlerror.interrm
Else Say 'CURL error:' curlerror.curlcode curlerror.curlerrm
Call CURLCleanup curl
Exit 1

This section will provide some common examples of the use of Rexx/CURL. ftp upload ] ftp download ] http get http post

History of Rexx/CURL

This section provides details of changes and additions made to the Rexx/CURL interface as it evolves.

Version 1.0: 4 Jul 2001

Initial release for Unix platforms.

Last updated 9 July 2001

A Rexx interface to the cURL library

Table of Contents

1. Introduction

2. Overview

3. Functions

4. Errors

5. Using Rexx/CURL

History of Rexx/CURL

Version 1.0: 4 Jul 2001