ParsedToken
in package
FinalYes
One token produced while parsing a text: a word or a non-word run (whitespace / punctuation).
This replaces the rows that used to be written to the temp_word_occurrences
scratch table during parsing. Tokens are produced fully in PHP and consumed
by TokenPersistence, so no scratch table is needed. See ScratchTables for the
history of why the old temp tables were removed.
Field meanings mirror the former temp_word_occurrences columns:
- $sentence: 1-based sentence index within this text (was TiSeID, but local, not a pre-computed SeID).
- $order: global monotonic token order across the whole text (was TiOrder).
- $wordCount: 1 for a word token, 0 for a non-word run (was TiWordCount).
- $text: the token text (was TiText).
Tags
Table of Contents
Properties
- $order : int
- $sentence : int
- $text : string
- $wordCount : int
Methods
- __construct() : mixed
- isWord() : bool
- Whether this token is a word (as opposed to whitespace/punctuation).
Properties
$order read-only
public
int
$order
$sentence read-only
public
int
$sentence
$text read-only
public
string
$text
$wordCount read-only
public
int
$wordCount
Methods
__construct()
public
__construct(int $sentence, int $order, int $wordCount, string $text) : mixed
Parameters
- $sentence : int
-
1-based sentence index within the text
- $order : int
-
Global monotonic token order across the text
- $wordCount : int
-
1 for a word, 0 for a non-word run
- $text : string
-
The token text
isWord()
Whether this token is a word (as opposed to whitespace/punctuation).
public
isWord() : bool