# KEHOME/doc/MKRtoken.html # Apr/4/2003 #========================# # MKR tokens # # version 5.4 # # # # Richard H. McCullough # #========================# # The MKR parser has three phases # # 1. token.icn: get_word(): identify low-level word classes, e.g. # b white space # s separator # w word # 2. token.icn: get_token(): identify special words and operators, e.g. # A at # H has isin # D do # o od # d done # R relation: isa isa* isa**n ... # P preposition: of with from to out # = assignment operator: =,+=,-=,*= # 3. symbol.icn: get_symbol(): identify phrases, sentences, ..., e.g. # www; word phrase # wHw=w; subject has attribute=value # wRw; subject rel object # wiwPw=w;subject is object with attribute=value #------------------------------------# # Here's the complete list of tokens # #------------------------------------# B white space: blank tab return newline newpage N non separator: ~ / \ ? . $ (included in w) W weak separator: + - * : (included in x) S strong separator: , ; = ! & | [ ] { } ( ) < > " ' # w word: consecutive bytes not B nor S nor W x extended word: consecutive w or W " quote mark: " or ' q quote # comment mark: # c comment # NOTE: # For high level parsing (phase 3 in symbol.icn), # the types above are remapped to # B => deleted # W => individual characters # S => individual characters # w => w # x => w # " => not used # q => w # # => not used # c => deleted , list separator: phrase ; list separator: sentence, proposition ~ OK in word: not : OK in word: view-concept, class-number, unit + OK in word - OK in word * OK in word: wildcard A at: context -- space,time,view C causes D do do* vdo d done E else F fi G generator: every,when,while g group: ntriple,mcf,rdf,... (use begin/end) H has isin h hierarchy (use begin/end) I if i is isa iss isu isd isi J conjunction: ";" iff implies causes means j conjunction: and or N complement: not non P preposition: out of with from to od at p haspart, isapart Q quantifier: a some the no all any either R relation: isa* isa** ... isc isc* isc** ... in ... r relation (use begin/end) S assignment: set vset unset s space T then t time U unknown v view w word not predefined by MKR Y group definition: begin Z group definition: end z exit () grouping: not currently used [] begin/end of comma list: phrase {} begin/end of semicolon list: proposition <> embedded HTML / down one level in hierarchy \ up one level in hierarchy ? question ! sh command $ value of variable $() product (a la KornShell) -- not currently used .. dot variable: current genus . dot variable: current concept ... dot variable: current species = assignment operator = = Set add operator += = Set delete operator -= = Set intersection operator *= = production operator := = Backus-Naur Form grammar operator ::= & and, asynchronous execution (not implemented) | or, pipe (not implemented) ~ not, login directory ##### possible additions ##### m method o od: direct object u unit