Samiksha, a student of class X was exploring the Natural Language Processing domain
Samiksha, a student of class X was exploring the Natural Language Processing domain. She got stuck while performing the text normalization. Help her to normalize the text on the segmented sentences given below:
Document 1: Akash and Ajay are best friends.
Document 2: Akash likes to play football but Ajay prefers to play online games.
Ans:
i. Tokenization:
Akash, and, Ajay, are, best, friends Akash, likes, to, play, football, but, Ajay, prefers, to, play, online, games
ii. Removal of stop words:
Akash, Ajay, best, friends Akash, likes, play, football, Ajay, prefers, play, online, games
iii. Converting text to a common case:
akash, ajay, best, friends akash, likes, play, football, ajay, prefers, play, online, games
iv. Stemming/Lemmatisation :
akash, ajay, best, friend akash, like, play, football, ajay, prefer, play, online, gam