modules/page/page_pluginmgr.man

[comment {-*- tcl -*- doctools manpage}]
[manpage_begin page_pluginmgr n 1.0]
[copyright {2007 Andreas Kupries <andreas_kupries@users.sourceforge.net>}]
[moddesc   {Parser generator tools}]
[titledesc {page plugin manager}]
[category  {Page Parser Generator}]
[keywords page {parser generator} {text processing}]
[require page::pluginmgr [opt 0.2]]
[require fileutil]
[description]
[para]

This package provides the plugin manager central to the [syscmd page]
application. It manages the various reader, writer, configuration, and
transformation plugins which actually process the text (read,
transform, and write).

[para]

All plugins are loaded into slave interpreters specially prepared for
them. While implemented using packages they need this special
environment and are not usable in a plain interpreter, like
tclsh. Because of that they are only described in general terms in
section [sectref {PREDEFINED PLUGINS}], and not documented as regular
packages. It is expected that they follow the APIs specified in the
sections

[list_begin enum]
[enum] [sectref {CONFIG PLUGIN API}]
[enum] [sectref {READER PLUGIN API}]
[enum] [sectref {WRITER PLUGIN API}]
[enum] [sectref {TRANSFORM PLUGIN API}]
[list_end]

as per their type.

[section API]

[list_begin definitions]

[call [cmd ::page::pluginmgr::reportvia] [arg cmd]]

This command defines the callback command used by

[cmd ::page::pluginmgr::report] (see below) to report input errors and
warnings. The default is to write such reports to the standard error
channel.

[call [cmd ::page::pluginmgr::report] [arg level] [arg text] [opt "[arg from] [opt [arg to]]"]]

This command is used to report input errors and warnings. By default
such reports are written to the standard error. This can be changed by
setting a user-specific callback command with
[cmd ::page::pluginmgr::reportvia] (see above).

[para]

The arguments [arg level] and [arg text] specify both the importance
of the message, and the message itself. For the former see the package
[package logger] for the allowed values.

[para]

The optional argument [arg from] and [arg to] can be used by the
caller to indicate the location (or range) in the input where the
reported problem occured. Each is a list containing two elements, the
line and the column in the input, in this order.


[call [cmd ::page::pluginmgr::log] [arg cmd]]

This command defines a log callback command to be used by loaded
plugins for the reporting of internal errors, warnings, and general
information. Specifying the empty string as callback disables logging.

[para]

Note: The [arg cmd] has to be created by the [package logger] package,
or follow the same API as such.

[para]

The command returns the empty string as its result.


[call [cmd ::page::pluginmgr::configuration] [arg name]]

This command loads the named configuration plugin, retrieves the
options encoded in it, and then immediately unloads it again.

[para]

If the [arg name] is the path to a file, then this files will be tried
to be loaded as a plugin first, and, if that fails, opened and its
contents read as a list of options and their arguments, separated by
spaces, tabs and newlines, possibly quotes with single and double
quotes.

[para]

See section [sectref {CONFIG PLUGIN API}] for the API expected of
configuration plugins.

[para]

The result of the command is the list of options retrieved.


[call [cmd ::page::pluginmgr::reader] [arg name]]

This command loads the named reader plugin and initializes it. The
result of the command is a list of options the plugin understands.

[para]

Only a single reader plugin can be loaded. Loading another reader
plugin causes the previously loaded reader plugin to be de-initialized
and unloaded.

[para]

See section [sectref {READER PLUGIN API}] for the API expected of
reader plugins.


[call [cmd ::page::pluginmgr::rconfigure] [arg dict]]

This commands configures the loaded reader plugin. The options and
their values are provided as a Tcl dictionary. The result of the
command is the empty string.

[call [cmd ::page::pluginmgr::rtimeable]]

This commands checks if the loaded reader plugin is able to collect
timing statistics. The result of the command is a boolean flag. The
result is [const true] if the plugin can be timed, and [const false]
otherwise.

[call [cmd ::page::pluginmgr::rtime]]

This command activates the collection of timing statistics in the
loaded reader plugin.

[call [cmd ::page::pluginmgr::rgettime]]

This command retrieves the collected timing statistics of the loaded
reader plugin after it was executed.

[call [cmd ::page::pluginmgr::rhelp]]

This command retrieves the help string of the loaded reader
plugin. This is expected to be in [term doctools] format.

[call [cmd ::page::pluginmgr::rlabel]]

This command retrieves the human-readable name of the loaded reader
plugin.

[call [cmd ::page::pluginmgr::read] [arg read] [arg eof] [opt [arg complete]]]

This command invokes the loaded reader plugin to process the input,
and returns the results of the plugin as its own result. The input is
accessible through the callback commands [arg read], and [arg eof]. The
optional [arg done] can be used to intrecept when the plugin has
completed its processing. All arguments are command prefixes.

[para]

The plugin will invoke the various callbacks in the following
situations:

[list_begin definitions]
[call [arg read] [arg num]]
is invoked whenever input to process is needed, with the number of
characters/bytes it asks for. The result is expected to be the input
the plugin is in need of.

[call [arg eof]]
is invoked by the plugin to check if the input has reached the of the
stream. The result is expected to be a boolean flag, [const true] when
the input has hit EOF, and [const false] otherwise.

[call [arg done]]
is invoked when the plugin has completed the processing of the input.

[list_end]

[call [cmd ::page::pluginmgr::writer] [arg name]]

This command loads the named writer plugin and initializes it. The
result of the command is a list of options the plugin understands.

[para]

Only a single reader plugin can be loaded. Loading another reader
plugin causes the previously loaded reader plugin to be de-initialized
and unloaded.

[para]

See section [sectref {WRITER PLUGIN API}] for the API expected of
writer plugins.

[call [cmd ::page::pluginmgr::wconfigure] [arg dict]]

This commands configures the loaded writer plugin. The options and
their values are provided as a Tcl dictionary. The result of the
command is the empty string.

[call [cmd ::page::pluginmgr::wtimeable]]

This commands checks if the loaded writer plugin is able to measure
execution times. The result of the command is a boolean flag. The
result is [const true] if the plugin can be timed, and [const false]
otherwise.

[call [cmd ::page::pluginmgr::wtime]]

This command activates the collection of timing statistics in the
loaded writer plugin.

[call [cmd ::page::pluginmgr::wgettime]]

This command retrieves the collected timing statistics of the loaded
writer plugin after it was executed.

[call [cmd ::page::pluginmgr::whelp]]

This command retrieves the help string of the loaded writer
plugin. This is expected to be in [term doctools] format.

[call [cmd ::page::pluginmgr::wlabel]]

This command retrieves the human-readable name of the loaded writer
plugin.

[call [cmd ::page::pluginmgr::write] [arg chan] [arg data]]

The loaded writer plugin is invoked to generate the output. It is
given the [arg data] to generate the outpout from, and the Tcl handle
[arg chan] of the channel to write the generated output to. The
command returns th empty string as its result.

[call [cmd ::page::pluginmgr::transform] [arg name]]

This command loads the named transformation plugin and initializes
it. The result of the command is a 2-element list containing the
plugin id and a list of options the plugin understands, in this order.

[para]

Multiple transformations plugins can be loaded and are identified by
handles.

[para]

See section [sectref {TRANSFORM PLUGIN API}] for the API expected of
transformation plugins.

[call [cmd ::page::pluginmgr::tconfigure] [arg id] [arg dict]]

This commands configures the identified transformation plugin. The
options and their values are provided as a Tcl dictionary. The result
of the command is the empty string.

[call [cmd ::page::pluginmgr::ttimeable] [arg id]]

This commands checks if the identified transformation plugin is able
to collect timing statistics. The result of the command is a boolean
flag. The result is [const true] if the plugin can be timed, and
[const false] otherwise.

[call [cmd ::page::pluginmgr::ttime] [arg id]]

This command activates the collection of timing statistics in the
identified transformation plugin.

[call [cmd ::page::pluginmgr::tgettime] [arg id]]

This command retrieves the collected timing statistics of the
identified transformation plugin after it was executed.

[call [cmd ::page::pluginmgr::thelp] [arg id]]

This command retrieves the help string of the identified
transformation plugin. This is expected to be in [term doctools]
format.

[call [cmd ::page::pluginmgr::tlabel] [arg id]]

This command retrieves the human-readable name of the identified
transformation plugin.

[call [cmd ::page::pluginmgr::transform_do] [arg id] [arg data]]

The identified transformation plugin is invoked to process the
specified [arg data]. The result of the plugin is returned as the
result of the command.

[list_end]

[section {CONFIG PLUGIN API}]

Configuration plugins are expected to provide a single command,
described below.

[para]

[list_begin definitions]
[call [cmd page_cdefinition]]

This command of a configuration plugin is called by the plugin manager
to execute it. Its result has to be a list of options and values to
process.

[list_end]
[para]

Configuration plugins do not expect the environment to provide any
special commands.

[para]

It is expected that a configuration plugin [const FOO] is implemented
by the package [package page::config::[const FOO]].

[para]

Configuration plugins are loaded, executed, and unloaded in one step,
they are not kept in memory. The command for doing this is
[cmd ::page::pluginmgr::configuration].

[section {READER PLUGIN API}]

Reader plugins are expected to provide the following commands,
described below.

[para]

[list_begin definitions]

[call [cmd page_rfeature] [arg name]]

This command takes a feature [arg name] and returns a boolean flag
indicating whether the feature is supported by the plugin, or not.

The result has to be [const true] if the feature is supported, and
[const false] otherwise.

[para]

See section [sectref FEATURES] for the possible features the plugin
manager will ask for.

[call [cmd page_rtime]]

This command is invoked to activate the collection of timing
statistics.

[call [cmd page_rgettime]]

This command is invoked to retrieve the collected timing statistics.

[call [cmd page_rlabel]]

This command is invoked to retrieve a human-readable label for the
plugin.

[call [cmd page_rhelp]]

This command is invoked to retrieve a help text for plugin. The text
is expected to be in [term doctools] format.

[call [cmd page_roptions]]

This command is invoked to retrieve the options understood by the
plugin.

[call [cmd page_rconfigure] [arg option] [arg value]]

This command is invoked to reconfigure the plugin, specifically the
given [arg option] is set to the new [arg value].

[call [cmd page_rrun]]

This command is invoked to process the input stream per the current
plugin configuration.  The result of the command is the result of the
processing.

[list_end]
[para]

Reader plugins expect the environment to provide the following special
commands.

[list_begin definitions]

[call [cmd page_read] [arg num]]

This command is invoked to read [arg num] characters/bytes from the
input. Its result has to be read characters/bytes.

[call [cmd page_read_done]]

This command is invoked to signal that the plugin has completed the
processing of the input.

[call [cmd page_eof]]

This command is invoked to check if the input stream has reached its
end. Its result has to be a boolean flag, [const true] when the input
has reached the end, [const false] otherwise.

[call [cmd page_info] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report some information to the user. May indicate a
location or range in the input. Each piece of location data, if
provided, is a 2-element list containing line and column numbers.

[call [cmd page_warning] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report a warning to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_error] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report an error to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_log_info] [arg text]]

Invoked to report some internal information.

[call [cmd page_log_warning] [arg text]]

Invoked to report an internal warning.

[call [cmd page_log_error] [arg text]]

Invoked to report an internal error.

[list_end]
[para]

It is expected that a reader plugin [const FOO] is implemented
by the package [package page::reader::[const FOO]].

[para]

Reader plugins are loaded by the command
[cmd ::page::pluginmgr::reader]. At most one reader plugin can be kept
in memory.

[section {WRITER PLUGIN API}]

Writer plugins are expected to provide the following commands,
described below.

[para]

[list_begin definitions]

[call [cmd page_wfeature]]

This command takes a feature [arg name] and returns a boolean flag
indicating whether the feature is supported by the plugin, or not.

The result has to be [const true] if the feature is supported, and
[const false] otherwise.

[para]

See section [sectref FEATURES] for the possible features the plugin
manager will ask for.

[call [cmd page_wtime]]

This command is invoked to activate the collection of timing
statistics.

[call [cmd page_wgettime]]

This command is invoked to retrieve the collected timing statistics.

[call [cmd page_wlabel]]

This command is invoked to retrieve a human-readable label for the
plugin.

[call [cmd page_whelp]]

This command is invoked to retrieve a help text for plugin. The text
is expected to be in [term doctools] format.

[call [cmd page_woptions]]

This command is invoked to retrieve the options understood by the
plugin.

[call [cmd page_wconfigure] [arg option] [arg value]]

This command is invoked to reconfigure the plugin, specifically the
given [arg option] is set to the new [arg value].

[call [cmd page_wrun] [arg chan] [arg data]]

This command is invoked to process the specified [arg data] and write
it to the output stream [arg chan]. The latter is a Tcl channel handle
opened for writing. The result of the command is the empty string.

[list_end]
[para]

Writer plugins expect the environment to provide the following special
commands.

[list_begin definitions]

[call [cmd page_info] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report some information to the user. May indicate a
location or range in the input. Each piece of location data, if
provided, is a 2-element list containing line and column numbers.

[call [cmd page_warning] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report a warning to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_error] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report an error to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_log_info] [arg text]]

Invoked to report some internal information.

[call [cmd page_log_warning] [arg text]]

Invoked to report an internal warning.

[call [cmd page_log_error] [arg text]]

Invoked to report an internal error.

[list_end]
[para]

It is expected that a writer plugin [const FOO] is implemented
by the package [package page::writer::[const FOO]].

[para]

Writer plugins are loaded by the command
[cmd ::page::pluginmgr::writer]. At most one writer plugin can be kept
in memory.

[section {TRANSFORM PLUGIN API}] page::transform::*

Transformation plugins are expected to provide the following commands,
described below.

[para]

[list_begin definitions]

[call [cmd page_tfeature]]

This command takes a feature [arg name] and returns a boolean flag
indicating whether the feature is supported by the plugin, or not.

The result has to be [const true] if the feature is supported, and
[const false] otherwise.

[para]

See section [sectref FEATURES] for the possible features the plugin
manager will ask for.

[call [cmd page_ttime]]

This command is invoked to activate the collection of timing
statistics.

[call [cmd page_tgettime]]

This command is invoked to retrieve the collected timing statistics.

[call [cmd page_tlabel]]

This command is invoked to retrieve a human-readable label for the
plugin.

[call [cmd page_thelp]]

This command is invoked to retrieve a help text for plugin. The text
is expected to be in [term doctools] format.

[call [cmd page_toptions]]

This command is invoked to retrieve the options understood by the
plugin.

[call [cmd page_tconfigure] [arg option] [arg value]]

This command is invoked to reconfigure the plugin, specifically the
given [arg option] is set to the new [arg value].

[call [cmd page_trun] [arg chan] [arg data]]

This command is invoked to process the specified [arg data] and write
it to the output stream [arg chan]. The latter is a Tcl channel handle
opened for writing. The result of the command is the empty string.

[list_end]
[para]

Transformation plugins expect the environment to provide the following
special commands.

[list_begin definitions]

[call [cmd page_info] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report some information to the user. May indicate a
location or range in the input. Each piece of location data, if
provided, is a 2-element list containing line and column numbers.

[call [cmd page_warning] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report a warning to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_error] [arg text] [opt "[arg from] [opt [arg to]]"]]

Invoked to report an error to the user. May indicate a location or
range in the input. Each piece of location data, if provided, is a
2-element list containing line and column numbers.

[call [cmd page_log_info] [arg text]]

Invoked to report some internal information.

[call [cmd page_log_warning] [arg text]]

Invoked to report an internal warning.

[call [cmd page_log_error] [arg text]]

Invoked to report an internal error.

[list_end]
[para]

It is expected that a transformation plugin [const FOO] is implemented
by the package [package page::transform::[const FOO]].

[para]

Transformation plugins are loaded by the command

[cmd ::page::pluginmgr::transform]. More than one transformation
plugin can be kept in memory.

[section {PREDEFINED PLUGINS}]

The following predefined plugins are known, i.e. provided by the page
module.

[list_begin definitions]

[def Configuration]
[list_begin definitions]
[def peg]

Returns a set of options to configure the [syscmd page] application
for the processing of a PEG grammar and the generation of ME code. See
the packages [package grammar_peg], [package grammar_me] and relations
for more details.

[list_end]

[def Reader]
[list_begin definitions]
[def hb]
Expects a so-called [term {half-baked PEG container}] as input and
returns the equivalent abstract syntax tree. See the writer plugin
[term hb] for the plugin generating this type of input.

[def lemon]
Expects a grammar specification as understood by Richar Hipp's LEMON
parser generator and returns an abstract syntax tree for it.

[def peg]
Expects a grammar specification in the form of a parsing expression
grammar (PEG) and returns an abstract syntax tree for it.

[def ser]
Expect the serialized form of a parsing expression grammar as
generated by the package [package grammar::peg] as input, converts it
into an equivalent abstract syntax tree and returns that.

[def treeser]
Expects the serialized form of a tree as generated by the package
[package struct::tree] as input and returns it, after validation.

[list_end]

[def Writer]
[list_begin definitions]
[def hb]
Expects an abstract syntax tree for a parsing expression grammar as
input and writes it out in the form of a so-called
[term {half-baked PEG container}].

[def identity]
Takes any input and writes it as is.

[def mecpu]
Expects symbolic assembler code for the MatchEngine CPU (See the
package [package grammar::me::cpu] and relatives) and writes it out as
Tcl code for a parser.

[def me]
Expects an abstract syntax tree for a parsing expression grammar as
input and writes it out as Tcl code for the MatchEngine (See the
package [package grammar::me] and relatives) which parses input in
that grammar.

[def null]
Takes any input and writes nothing. The logical equivalent of
/dev/null.

[def peg]
Expects an abstract syntax tree for a parsing expression grammar as
input and writes it out in the form of a canonical PEG which can be
read by the reader plugin [term peg].

[def ser]
Expects an abstract syntax tree for a parsing expression grammar as
input and writes it out as a serialized PEG container which can be
read by the reader plugin [term ser].

[def tpc]
Expects an abstract syntax tree for a parsing expression grammar as
input and writes it out as Tcl code initializing a PEG container as
provided by the package [package grammar::peg].

[def tree]
Takes any serialized tree (per package [package struct::tree]) as
input and writes it out in a generic indented format.

[list_end]

[def Transformation]
[list_begin definitions]

[def mecpu]
Takes an abstract syntax tree for a parsing expression grammer as
input, generates symbolic assembler code for the MatchEngine CPU, and
returns that as its result (See the package [package grammar::me::cpu]
and relatives).

[def reachable]
Takes an abstract syntax tree for a parsing expression grammer as
input, performs a reachability analysis, and returns the modified and
annotated tree.

[def realizable]
Takes an abstract syntax tree for a parsing expression grammer as
input, performs an analysis of realizability, and returns the modified
and annotated tree.

[list_end]
[list_end]

[comment {
	Make it clear that all data between plugins is shuffled around
	in serialized form, as objects cannot be transfered/accessed
	across the interpreter boundaries (safety concerns).

	Describe the commands expected by plugins to be available in
	the environment.

	Describe the predefined features.
}]


[section FEATURES]

The plugin manager currently checks the plugins for only one feature,
[const timeable]. A plugin supporting this feature is assumed to be
able to collect timing statistics on request.


[section {BUGS, IDEAS, FEEDBACK}]

This document, will undoubtedly contain bugs and other problems.

Please report such in the category [emph page] of the
[uri {http://sourceforge.net/tracker/?group_id=12883} {Tcllib SF Trackers}].

Please also report any ideas for enhancements you may have.

[manpage_end]