forked from ajaxorg/ace
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathtokenizer.html
More file actions
122 lines (116 loc) · 7.71 KB
/
tokenizer.html
File metadata and controls
122 lines (116 loc) · 7.71 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
<div id="documentation" class="span9">
<div class="classContent">
<div class="membersBackground"></div>
<div class=" members pos0">
<div class=" membersContent pos0">
<h1 class="memberHeader"><span class="name">Tokenizer</span>
</h1>
<ul data-tabs="tabs" class="nav tabs pos0">
<li data-dropdown="dropdown" class="dropdown">
<ul class="dropdown-menu">
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Constructors (1)</span><b class="caret"></b></a>
<ul class="dropdown-menu">
<li id="dropdown_Tokenizer.new" data-id="Tokenizer.new" class="memberLink"><a href="#Tokenizer.new" class="" title="Tokenizer.new (constructor)" data-id="Tokenizer.new">new</a>
</li>
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Functions (1)</span><b class="caret"></b></a>
<ul class="dropdown-menu">
<li id="dropdown_Tokenizer.getLineTokens" data-id="Tokenizer.getLineTokens" class="memberLink"><a href="#Tokenizer.getLineTokens" class="" title="Tokenizer.getLineTokens (class method)" data-id="Tokenizer.getLineTokens">getLineTokens</a>
</li>
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown">
<ul class="dropdown-menu">
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown">
<ul class="dropdown-menu">
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown">
<ul class="dropdown-menu">
</ul>
</li>
<li data-dropdown="dropdown" class="dropdown">
<ul class="dropdown-menu">
</ul>
</li>
</ul>
</div>
</div>
<article id="Tokenizer" data-title="Tokenizer (class)" class="article">
<div class="section description">
<div class="memberContent"><p>This class takes a set of highlighting rules, and creates a tokenizer out of them. For more information, see <a href="https://github.com/ajaxorg/ace/wiki/Creating-or-Extending-an-Edit-Mode#wiki-extendingTheHighlighter">the wiki on extending highlighters</a>.</p>
</div>
</div>
</article>
<h3 class="sectionHeader">Constructors</h3>
<article id="Tokenizer.new" data-title="Tokenizer.new (constructor)" class="article">
<div class="section method">
<div class="memberContent">
<div class="title"><i id="Tokenizer.new" class="methodToggle methodClicker inactive icon-caret-right"></i>
<ul class="signatures">
<li class="signature">
<ul>
<li class="signature-call"><span class="constructorIdentifier">new </span><span id="Tokenizer.new" class="member-name methodClicker">Tokenizer</span>(<span class="sigArgList"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> rules, <a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String" class="argument methodClicker" title="String" data-id="String">String</a> flag</span>)</li>
</ul>
<ul class="metaInfo">
</ul>
</li>
</ul>
</div>
<div class="sideToggler">
<div id="ellipsis_Tokenizer.new" class="ellipsis_description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
</div>
<div class="description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
<h4>Arguments</h4><table class="argumentTable argument-list table table-striped table-bordered"><tr class="argumentRow "><td class="argName ">rules</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. The highlighting rules</p>
</td></tr><tr class="argumentRow "><td class="argName ">flag</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String" class="" title="String" data-id="String">String</a></td><td class="argDescription "><p>Required. Any additional regular expression flags to pass (like "i" for case insensitive)</p>
</td></tr></table>
</div>
</div>
</div>
</div>
</article>
<h3 class="sectionHeader">Methods</h3>
<article id="Tokenizer.getLineTokens" data-title="Tokenizer.getLineTokens (class method)" class="article">
<div class="section method">
<div class="memberContent">
<div class="title"><i id="Tokenizer.getLineTokens" class="methodToggle methodClicker inactive icon-caret-right"></i>
<ul class="signatures">
<li class="signature">
<ul>
<li class="signature-call"><span id="Tokenizer.getLineTokens" class="member-name methodClicker"><span class="sigClassName">Tokenizer.</span><span class="sigMemberName">getLineTokens</span></span>(<span class="sigArgList"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> line, <a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> startState</span>)
<li class="signature-returns">
<ul class="argument-types">
<li class="argument-type"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="returnType " title="Object" data-id="Object">Object</a></li>
</ul>
</li></li>
</ul>
<ul class="metaInfo">
</ul>
</li>
</ul>
</div>
<div class="sideToggler">
<div id="ellipsis_Tokenizer.getLineTokens" class="ellipsis_description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
</div>
<div class="description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
<h4>Arguments</h4><table class="argumentTable argument-list table table-striped table-bordered"><tr class="argumentRow "><td class="argName ">line</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. </p>
</td></tr><tr class="argumentRow "><td class="argName ">startState</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. </p>
</td></tr></table>
</div>
</div>
</div>
</div>
</article>
</div>
<script defer src="./resources/javascripts/ux.js"></script>
<script src="./resources/javascripts/clicker.js"></script>
<script src="./resources/javascripts/jquery-scrollspy.js"></script>
<script defer src="./resources/javascripts/disqus-ext.js"></script>
<script defer src="./resources/javascripts/ga.js"></script>
<div id="disqus_thread"></div>
</div>