The Art of Tokenization in Text Preprocessing: Day2
Exploring Basic to Advanced NLP Tokenization Techniques
15 min readNov 18, 2023
Table of Contents: We will cover the following
· Introduction
· What is Tokenization?
· Types of Tokenization
· Basic Tokenization Types
· Subword Tokenization
· Specialized Tokenization
· Advanced Tokenization
· Conclusion
· What’s Coming Up Next?