Tokenize raw text by scanning characters into words, punctuation marks, and supported special token forms.
