pyab_experiment.language.lexer

Lexical Analysis for Experiment Language Definition

This module defines the lexer for the experiment language definition. It includes common constructs such as operators, identifiers, and literals (floats, integers). The module also defines reserved keywords used by the grammar.

The lexical analysis is performed using YACC (via the SLY implementation) for finite state automaton processing.

Module Contents

Classes

BlockComment	Helper state that deals with C style opening
ExperimentLexer	Lexical analyzer for an AB testing configuration language.

class pyab_experiment.language.lexer.BlockComment

Bases: pyab_experiment.sly.Lexer

Helper state that deals with C style opening and closing block comments

ignore_newline = '\\n+'

tokens

BLOCK_COMMENT_END(t)

ignore_newline(t)

t_block_comment_content(t)

class pyab_experiment.language.lexer.ExperimentLexer

Bases: pyab_experiment.sly.Lexer

Lexical analyzer for an AB testing configuration language.

This class provides a method for tokenizing an AB testing configuration file into a list of tokens, to be used for parsing and validation, in conjunction with the grammar defined in a companion class. The lexer uses regular expressions to match terminal patterns in the configuration file and generate corresponding tokens.

COLON = ':'

COMMA = ','

ID = '[a-zA-Z_][a-zA-Z0-9_]*'

KW_AND = 'and'

KW_DEF = 'def'

KW_ELIF = 'else\\s*if'

KW_ELSE = 'else'

KW_EQ = '=='

KW_GE = '>='

KW_GT = '>'

KW_IF = 'if'

KW_IN = 'in'

KW_LE = '<='

KW_LT = '<'

KW_NE = '!='

KW_NOT = 'not'

KW_NOT_IN = 'not\\s+in'

KW_OR = 'or'

KW_RETURN = 'return'

KW_SALT = 'salt'

KW_SPLITTERS = 'splitters'

KW_WEIGHTED = 'weighted'

LBRACE = '{'

LPAREN = '\\('

MINUS = '-'

RBRACE = '}'

RPAREN = '\\)'

ignore_inline_comment = '//.*'

ignore_newline = '\\n+'

ignore_ws = '\\s+'

tokens

BLOCK_COMMENT_START(t)

NON_NEG_FLOAT(t)

NON_NEG_INTEGER(t)

STRING_LITERAL(t)

error(t)

ignore_newline(t)