From Tokenization to Digitization: Explaining the Shift