Perl text document stats script

Crikey, that was a little easy, in less than a fortnight I've done a cover version of my Python script from here, now in Perl, all this from not knowing any Perl at all.

I didn't do it on my own mind, I had a textbook, Learning Perl from O'Reilly, and tutoring from Robbie and Matt off of Facebook, without their help, I'd be floored.

The program in perl that analyzes the text file of my 2001 novel Shag Times and comes up with various statistics for it.

Its a bit messy compared to the Python version, and its clunky too, there's gotta be simpler and more leet ways of doing many of the sections.

#!/usr/bin/perl

# Program to
#  open shagtimes
#  provide a word count              
#  count unique words                
#  provide top ten most popular words
#  provide all single occurance words
#  calculate average word length     
#  find longest word                 

use Text::Wrap;

sub hashValueDescendingNum {
  $occurance_list{$b} <=> $occurance_list{$a};
}

# Opening Shag Times and processing it a bit
$filename = "shagtimes.txt";
open BOOK, "<", $filename or die "Can't open '$filename': $!"; my @book = ;
close BOOK;
foreach $book (@book) {
 $allbook .= $book;
 $allbook .= " "};
$allbook = lc"$allbook";
# $allbook is a long string of Shag Times

# Code to remove full stops and commas and stuff
$regexp = '[\W]' ;
@book = split /\s+/, $allbook;
foreach $book (@book) {
 $book =~ s/$regexp//g };
print "\n=======================================\n";

# Doing the word count
foreach $book (@book) {
   $wordcount += 1 };
print "The document contains $wordcount words in total\n";
print "=======================================\n";

# Doing the unique count
foreach $book (@book) {
 $was_this_in_uniques = "no";
 foreach $uniques ( @uniques ) {
   if ($book eq $uniques){
     $was_this_in_uniques = "yes"}};
 if ($was_this_in_uniques eq "no") {
   push @uniques, $book}};
foreach $uniques (@uniques) {
   $uniquecount += 1 };
print "The document contains $uniquecount unique words\n";
print "=======================================\n";

# Finding top ten popular words
print "The top ten most used words:-\n";
foreach $uniques ( @uniques ) {
 $was_this_in_book = "no";
 foreach $book ( @book ) {
   if ($uniques eq $book) {
     $occurances += 1}};
 $occurance_list{$uniques} = $occurances;
 $occurances = 0 };
foreach $key (sort hashValueDescendingNum (keys(%occurance_list))) {
  push @ranked_occurances, "$key \($occurance_list{$key}\)"};
foreach (0..9) {
 print "$ranked_occurances[$_]\n"};
print "=======================================\n";

# Finding single use words
print "Words that were used only once:-\n";
foreach $ranked_occurances ( @ranked_occurances ) {
 $_ = $ranked_occurances;
 if (/\(1\)/) {
   s/\s\(1\)//;
   push @singles, ("$_");
   $single_use_count += 1}};
@singles = sort @singles;
foreach $singles ( @singles ) {
 $single_paragraph .= "$singles, "};
print wrap("", "", "$single_paragraph\n");
print "=======================================\n";
print "A total of $single_use_count words were used only once\n";
print "=======================================\n";

# Finding average word length
foreach $book (@book) {
   $chartotal += length($book) };
$avechar = $chartotal/$wordcount;
my $printy_avechar = sprintf "%.3f", $avechar;
print "The average word length was $printy_avechar letters long\n";
print "=======================================\n";

# Finding longest word
foreach $uniques ( @uniques ) {
 if (length($uniques) > $longlength) {
   $longlength = length($uniques)}};
print "The longest word was $longlength letters long\n";
print "These words were that long:-\n";
foreach $uniques ( @uniques ) {
 if (length($uniques) == $longlength) {
   print "$uniques\n"}};
print "=======================================\n";

Now I need to order that Ruby book, Ruby on the Rails for Dummies perhaps?

Python text document stats script

Desperately I spend the last three weeks refreshing my knowledge of Python, the easy peasy scripting language I'd once used in my youth to test CD players and hi-fi RS232 commands. I re-acquired Dive Into Python and Python For Dummies and ploughed through.

Sadly I didn't get the job I was learning it for, but instead refired my enthusiasm for learning programming languages. So, I've acquired Learning Perl and when that is done I'll be trying my hand at Ruby, just like the cool kids use.

I wrote me a wee program in Python that analyzes the text file of my 2001 novel Shag Times and comes up with various statistics for it.

I plan to re-write the same program in Perl as soon as I've finished reading Learning Perl, and then write the same program again in Ruby if that's even possible.

### Program to do the following
###  * open shagtimes.txt
###  * provide a word count
###  * count unique words        
###  * provide top ten most popular words
###  * provide all single occurance words
###  * calculate average word length
###  * find longest word         

import textwrap

# Opening Shag Times and processing it a bit
book = open('shagtimes.txt')
book = book.read()
book = book.lower()
### Code to remove punctuation
stuff_to_replace_with_space = (".", ",", "?", "/", "=", "-", ";", ":")
stuff_to_remove = ("\'", "(", ")", "\"")
for item in stuff_to_replace_with_space:
book = book.replace(item, " ")
for item in stuff_to_remove:
book = book.replace(item, "")
print "======================================="

# Doing the word count
book = book.split()
wordcount = len(book)
print "The document contains %i words in total" % wordcount
print "======================================="

# Doing the unique count
uniques = []
for item in book:
if item not in uniques:
uniques.append(item)
uniquecount = len(uniques)
print "The document contains %i unique words" % uniquecount
print "======================================="
uniques.sort()

# Finding top ten popular words
print "The top ten most used words:-"
occurancelist = []
for item in uniques:
occurances = book.count(item)
occurancelist.append((occurances, item))
occurancelist.sort()
occurancelist.reverse()
for item in occurancelist[:10]:
print item
print "======================================="

# Finding single use words
print "Words that were used only once:-"
singleuse = []
singleusecount = 0
for item in occurancelist:
c, w = item
if c == 1:
singleuse.append(w)
singleusecount+=1
singles = ""
while singleuse:
for item in singleuse:
singles = singles + (singleuse.pop()) + ", "
singles = textwrap.wrap(singles, width=70)
for i in singles:
print i
print "======================================="
print "A total of %i words were used only once" % singleusecount
print "======================================="

# Finding average word length
chartotal = 0.000
for item in book:
chartotal = chartotal + len(item)
avechar = chartotal/wordcount
print "The average word length was %.3f letters long" % avechar
print "======================================="

# Finding longest word
longlength = 0
for item in uniques:
if len(item) > longlength:
longlength = len(item)
print "The longest word was %i letters long" % longlength
print "These words were that long:-"
for item in uniques:
if len(item) == longlength:
print item
print "======================================="

Can someone recommend me a good book for learning to program in Ruby?