can generate context-free languages

require only single operand

require two operands

used as the general terminology for naming of variables, functions and arrays

formal language used to communicate algorithms both from programmer to programmer and from programmer to machine

refer to fixed values

using the particular constructs of the language

match a single character

always enclosed in double quotes

sequence of characters (alphanumeric) in a token

their values can't be modified by the program once they are defined

match zero or more quantifier

notations for describing a set of character strings

set of symbols and rules for forming terms

takes the modified source code from language preprocessors that are written in the form of sentences

used as range quantifiers

special symbols

define as a set of strings of symbols that may be constrained by specific rules

also called as literals

create pointer variable

basic abstract unit of meaning

used for grouping characters or regular expressions

The rules for transforming terms to terms

used as array element reference

used to separate more than one statements

symbols that trigger an action when applied to variables and other objects

used in search engines, search and replace dialogues of word processors

breaks these syntaxes into a series of tokens by removing white space or comments in the source code

used to indicate function calls and function parameters

Mark the start and end of a block code

an array of characters ended with a null character

match 0 or 1 quantifier

indicates a range in character class

operator that essentially invokes an initialization list

separates a series of alternatives

anchor for the start of the string or negation symbol

requires three operands

anchor for the end of the string

gives special meaning to the character following it

an object that represents something else

predefined or reserved words in a programming language

some predefined rules for every lexeme to be identified as a valid token

match one or more quantifier

matches any character except the new line symbol

first phase of a compiler

different types of tokens

used in programming languages having some special meaning