NAME

sgmltoken - a sample LT NSL program for tokenising the text in an

SYNOPSIS

usage: sgmltoken [-d ddb-file] [-u base-url] [input-file]

DESCRIPTION

The material below may be out of date: consult LT XML documentation please.

The input file to sgmltoken is an nSGML file which contains <TEXT...> and <P> elements. All text inside such <TEXT...> elements will be tokenised into 'words' and punctuation represented by <C> elements.

DESCRIPTION: Input/Output

Description of the input/output files involved in this program.

Input ==> An nSGML file : [<filename> or stdin]

Output ==> An nSGML file containing <C> elements for the 'words',

in addition to the existing markup: [stdout]

EXPECTED DOCTYPE of INPUT

The <!DOCTYPE> for the input file should contain at least the following:
<!element text - - (#PCDATA|c|w|s|p)*>
<!element p - - (#PCDATA|c|w|s)*>
<!element s - - (w)*>
<!attlist s
id ID #IMPLIED>
<!element w - - (c)*>
<!attlist w
id ID #IMPLIED
type CDATA #IMPLIED
lemma CDATA #IMPLIED>
<!element c - - (#PCDATA)*>
<!attlist c
id ID #IMPLIED
rend CDATA #IMPLIED>

OPTIONS

-d <ddbfile>

is the name of a file containing a representation of a DTD. Can be used if the DTD is not specified (in a <?NSL DDB ...> statement) in the input document iself.

SEE ALSO

ltxml(1), mknsg(1), sgmlsb(1)

AUTHOR

Henry Thompson (ht@cogsci.ed.ac.uk)
David McKelvie (dmck@cogsci.ed.ac.uk)
Language Technology Group, Human Communication Research Centre, Edinburgh University,
2 Buccleuch Place, Edinburgh EH8 9LW, SCOTLAND
Tel:(44) 131 650-4630
Fax:(44) 131 650-4587 email: dmck@cogsci.ed.ac.uk

Comments, suggestions, and bug reports are always welcome.