TextParsingPersistence
in package
Shared database operations for text parsing.
Handles SQL-based text saving, sentence/text-item registration, statistics display, and multi-word expression checking.
Tags
Table of Contents
Methods
- checkExpressions() : void
- Check a language that contains expressions.
- checkValid() : void
- Echo the sentences in a text. Prepare JS data for words and word count.
- displayStatistics() : void
- Display statistics about a text.
- getMultiWordLengths() : array<string|int, int>
- Get all multi-word expression lengths for a language.
- registerSentencesTextItems() : void
- Append sentences and text items in the database.
- saveWithSql() : void
- Insert a processed text in the data in pure SQL way.
- saveWithSqlFallback() : void
- Fallback method to insert text data when LOAD DATA LOCAL INFILE is disabled.
Methods
checkExpressions()
Check a language that contains expressions.
public
static checkExpressions(array<string|int, int> $wl) : void
Parameters
- $wl : array<string|int, int>
-
All the different expression length in the language.
checkValid()
Echo the sentences in a text. Prepare JS data for words and word count.
public
static checkValid(int $lid) : void
Parameters
- $lid : int
-
Language ID
displayStatistics()
Display statistics about a text.
public
static displayStatistics(int $lid, bool $rtlScript, bool $multiwords) : void
Parameters
- $lid : int
-
Language ID
- $rtlScript : bool
-
true if language is right-to-left
- $multiwords : bool
-
Display if text has multi-words
getMultiWordLengths()
Get all multi-word expression lengths for a language.
public
static getMultiWordLengths(int $lid) : array<string|int, int>
Parameters
- $lid : int
-
Language ID
Return values
array<string|int, int> —Array of distinct word counts (e.g., [2, 3] for 2-word and 3-word expressions)
registerSentencesTextItems()
Append sentences and text items in the database.
public
static registerSentencesTextItems(int $tid, int $lid, bool $hasmultiword) : void
TiSeID in temp_word_occurrences is pre-computed to match future SeID values. When parseStandardToDatabase runs with useMaxSeID=true, it sets TiSeID to MAX(SeID)+1, MAX(SeID)+2, etc. When we insert sentences here, they get those exact SeID values via auto-increment, so TiSeID = SeID.
Parameters
- $tid : int
-
ID of text from which insert data
- $lid : int
-
ID of the language of the text
- $hasmultiword : bool
-
Set to true to insert multi-words as well.
saveWithSql()
Insert a processed text in the data in pure SQL way.
public
static saveWithSql(string $text, int $id) : void
Parameters
- $text : string
-
Preprocessed text to insert
- $id : int
-
Text ID
saveWithSqlFallback()
Fallback method to insert text data when LOAD DATA LOCAL INFILE is disabled.
public
static saveWithSqlFallback(string $text, int $id) : void
Parameters
- $text : string
-
Preprocessed text to insert
- $id : int
-
Text ID