In this project we built a simple language model that can be used to generate random text resembling a source document. This program tokenizes the source texts and processes it to produce frequencies of words following a given n-gram. The model then uses these frequencies to determine the probablility of certain words occuring at a point in a document. With this information the model can generate random text that resembles the source document.
In this repoitory I've supplied some sample text from Hunter S. Thompson's Fear and Loathing in Las Vegas that can be used to generate random sentences that resemble the writing of Hunter S. Thompson
“Why what ? ” he said . “But let’s forget that bullshit about the Laos Invasion – a goddamn miracle if we didn’t get there by four to claim our soundproof suite . ” He was staring up at the toll - gate , too twisted to find the American Dream . ” “Use me ? ” he said . “And my first advice is that you should rent a very handsome man if you’d just stand up ! You’d be a very crude man . “We’re giving you a break . “I have the ticket . * * * But what now ? At 9 : 00 . There was a very crazy looking person , half - hidden parking lot of the traps every five seconds or so . Let’s carve up this grapefruit and other luggage out to grab him – but then , back off and bury him somewhere . Because we both had bad hearts ? “How about some ether ? ” By the time , I felt , to facilitate the tanning process . “What the fuck out of L . A . , pasties here – total naked public humping in L . A . – just a few hours ? That’s because he’s a foreigner . I thought . Sign my attorney shouted as I eased the car into the men’s room to eat mine . So the next gas station . …” He paused , listened for an instant , then poured some into a wild animal . Are you ready for the Mint 400 . ” Right . But that was some other era , burned out adrenalin reserves – a stainless - steel hunting knife in your eyes . Not for me . The warden will recognize me ; and the next five years of summer mornings playing basketball in the patio section , of course – just a neighbor with a sign saying “Max Speed 25” … and here I am . You’d go crazy if I stood up ! ” I nodded , barely able to talk ? I turned to the Mint 400 . ’ But officers handcuffed the woman and she apparently was not injured . …” And suddenly there was no sound or movement in the mesquite - desert , this man Thompson .