CS374: Programming Language Principles - Tokens and Scanning Lexemes
Activity Goals
The goals of this activity are:
- To define a lexeme and to define tokens based on lexemes
- To differentiate between types, reserved words, and identifiers in a scanner
- To read a file and identify defined tokens
The Activity
Directions
Consider the activity models and answer the questions provided. First reflect on these questions on your own briefly, before discussing and comparing your thoughts with your group. Appoint one member of your group to discuss your findings with the class, and the rest of the group should help that member prepare their response. Answer each question individually from the activity on the Class Activity Questions discussion board. After class, think about the questions in the reflective prompt and respond to those individually in your notebook. Report out on areas of disagreement or items for which you and your group identified alternative approaches. Write down and report out questions you encountered along the way for group discussion.
Model 1: Scanning Lexemes for Tokens with flex
Questions
- What is a lexeme? What is a token?
- Describe an algorithm to read a
String from left to right, returning tokens as they are identified.
- How would you modify this scanner configuration to support an assignment operator and an identifier (variable) token?
Submission
Submit your answers to the questions using the Class Activity Questions discussion board. You may also respond to questions or comments made by others, or ask follow-up questions there. Answer any reflective prompt questions in the Reflective Journal section of your OneNote Classroom personal section.