create_tokens {styler} | R Documentation |
Creates a terminal token represented as (a row of) a parse table.
create_tokens(tokens, texts, lag_newlines = 0, spaces = 0, pos_ids, token_before = NA, token_after = NA, indention_ref_pos_ids = NA, indents = 0, terminal = TRUE, child = NULL)
tokens |
Character vector with tokens to create. |
texts |
Character vector with texts of the token to create. |
lag_newlines |
Character vector with lag_newlines corresponding to the tokens. |
spaces |
Character vector with spaces corresponding to the tokens. |
pos_ids |
Character vector with positional id corresponding to the tokens. |
token_before |
Character vector corresponding to the columns
|
token_after |
Character vector corresponding to the columns
|
indention_ref_pos_ids |
Character vector with indention ref ids corresponding to the tokens. |
indents |
Vector with indents corresponding to the tokens. |
terminal |
Boolean vector indicating whether a token is a terminal or not. |
child |
The children of the tokens. |
Other token creators: create_pos_ids
,
validate_new_pos_ids