# KEHOME/doc/MKRtoken.html
# Mar/12/2003
#========================#
# MKR tokens #
# version 5.3 #
# #
# Richard H. McCullough #
#========================#
# The MKR parser has three phases
#
# 1. token.icn: get_word(): identify low-level word classes, e.g.
# b white space
# s separator
# w word
# 2. token.icn: get_token(): identify special words and operators, e.g.
# A at
# H has isin
# D do
# o od
# d done
# R relation: isa isa* isa**n ...
# P preposition: of with from to out
# = assignment operator: =,+=,-=,*=
# 3. symbol.icn: get_symbol(): identify phrases, sentences, ..., e.g.
# www; word phrase
# wHw=w; subject has attribute=value
# wRw; subject rel object
# wiwPw=w;subject is object with attribute=value
#------------------------------------#
# Here's the complete list of tokens #
#------------------------------------#
B white space: blank tab return newline newpage
N non separator: ~ / \ ? . $ (included in w)
W weak separator: + - * : (included in x)
S strong separator: , ; = ! & | [ ] { } ( ) < > " ' #
w word: consecutive bytes not B nor S nor W
x extended word: consecutive w or W
" quote mark: " or '
q quote
# comment mark: #
c comment
# NOTE:
# For high level parsing (phase 3 in symbol.icn),
# the types above are remapped to
# B => deleted
# W => individual characters
# S => individual characters
# w => w
# x => w
# " => not used
# q => w
# # => not used
# c => deleted
, list separator: phrase
; list separator: sentence, proposition
~ OK in word: not
: OK in word: view-concept, class-number, unit
+ OK in word
- OK in word
* OK in word: wildcard
A at: context -- space,time,view
C causes
D do do* vdo
d done
E else
F fi
G generator: every,when,while
g group: hierarchy,unithierarchy,relation,... use begin/end
H has isin
h HTML command
I if
i is isa iss isu isd isi
J conjunction: ";" iff implies causes means
j conjunction: and or
N complement: not non
P preposition: out of with from to od at
p haspart, isapart
Q quantifier: a some the no all any either
R relation: isa* isa** ... isc isc* isc** ... in ...
S assignment: set vset unset
T then
U unknown
w word not predefined by MKR
Y group definition: begin
Z group definition: end
z exit
() grouping: not currently used
[] begin/end of comma list: phrase
{} begin/end of semicolon list: proposition
<> embedded HTML
/ down one level in hierarchy
\ up one level in hierarchy
? question
! sh command
$ value of variable
$() product (a la KornShell) -- not currently used
.. dot variable: current genus
. dot variable: current concept
... dot variable: current species
= assignment operator =
= Set add operator +=
= Set delete operator -=
= Set intersection operator *=
= production operator :=
= Backus-Naur Form grammar operator ::=
& and, asynchronous execution (not implemented)
| or, pipe (not implemented)
~ not, login directory
##### possible additions #####
h hierarchy: the word
m method
o od: direct object
r relation: the word
s space
t time
u unit
v view