Journey on Rails #3

The volumes in ChaptersS have come in one big file: all books, chapters, and verses in one file.

Remember the format I was shooting for?

Chapter ##

1 This is verse one.

2 This is verse two.

The format the books came in was:

Book Name

1:1 This is chapter 1, verse 1.

1:2 This is chapter 1, verse 2.

2:1 This is chapter 2, verse 1.

2:2 This is chapter 2, verse 2.

and so on...

OK so here is my text processing journey:

Processing Attempt #1:

• Open volume file in text editor.
• Manually cut-paste-save individual books.
• Open individual book files:
• Manually cut-paste-save individual chapters. Filenames were "book##.txt" where "##" was a two digit representation of the chapter number padded where necessary.
• Manually create "Chapter ##" header. This was accomplished using a keyboard macro to try to speed things up.
• Use very simple regex search-replace to replace chapter:verse markers with just a verse.
• Use very simple regex search-replace to create paragraph tags around each verse.
• Use text editor to replace a single quotation mark/apostrophe in all the chapter files with a \' so that my SQL insert script would not break.
• Total Time: About 7 hours.
  

Very tedious. You would think I knew better. In college I had a job where I cleaned up and formatted scanned texts for the folio indexing software. How embarrassing.

Processing Attempt #2:

• Open volume file in text editor.
• Manually cut-paste-save individual books.
• Open individual book files:
• Use regex search-replace to create chapter headers for every chapter in the book.
• Use regex to replace the chapter:verse markers with just a verse.
• Use regex search-replace to create paragraph tags around the verses.
• Search and replace all single quotation marks/apostrophes with \'.
• Total Time: About 4 hours.

Not bad but still very time consuming.

Processing Attempt #3:

• The books were already broken out for this volume. I must have done it earlier and forgotten.
• This time I decided to write a Ruby script to do all the work for me. Not only would this be fun and educational but it would speed up my processing of future volumes as it would be re-usable.
• In summary my script does the following:
• Opens the individual book file.
• Figures out when a new chapter begins.
• Opens a new chapter file with the name "book##.txt" like before.
• Adds the chapter header line to the file ("Chapter ##").
• Replaces chapter:verse markers with just verse in every verse.
• Adds paragraph tags around every verse.
• Replaces single quotation marks/apostrophes with \'.
• Writes verses to file.
• Total Time: 2 hours 30 seconds (2 hours to create script, 30 seconds to run it)

Processing Attempt #4:

This hasn't happened yet. When it does then I will attempt to enhance my script to process an entire volume instead of individual book files. The only problem I foresee is that the book titles within the volume can be very long, like "The Gospel According to Matthew". There might be some manual pre-processing of the volume file before I can script it totally.

Getting the Chapters in the Database

I created a shell script that would process each chapter file and insert it with an appropriate title, volume_id and volumeOrder.

Whewww. Hopefully, the length of this post reflects the pain of processing the volume text files into usable content.

Next up: ChaptersS on Rails

Journey on Rails #2

My idea for a layout was to make it look like scriptures with rounded corners, etc. That took me a few tries but I eventually got it working with as simple of CSS/HTML as I could. Maybe there are easier and more elegant ways. My approach seems to work fine in Firefox and Safari.

My next things to work on were the code and the content. I attacked these at the same time. When I got tired of one I worked on the other. I have a feeling that the content will be an on-going thing. So far I have the Book of Mormon, The Old Testament, and The New Testament.

So far I have gotten my text from the Gutenberg project for free and have painstakingly formatted it for my needs.

I have a volumes table and a chapter table. Each chapter is a row in the chapter table and is stored along with its volume number and the numeric order of the chapter within the volume. This sequential order number is important to supplying the feeds. Each chapter also has a short title. I format each chapter like this:

Chapter ##

1 This is verse one.

2 This is verse two.

The HTML for the chapters looks like this:
<p><b>Chapter ##</b></p>

<p>1 This is verse one.</p>

<p>2 This is verse two.</p>

I formatted things this way so that the same content stored in the chapters table could be used not only to look pretty in an RSS feed reader but also as an HTML page if someone wanted to read straight from the application instead of via RSS.

Journey on Rails #1

Several months ago I started teaching myself Ruby on Rails. I had some experience with PHP and MySQL and thought it was kind of mechanical. When I heard about Rails I thought it sounded, well, beautiful or at least a lot less mechanical. I bought the beta Agile Rails 2nd Edition book and started going through it. That was fun but I didn't need a shopping cart. I wanted to write a "real" application. And I thought documenting it might be fun too.



I had several ideas for applications going through my head and the one that was going to be the most immediately useful was also going to be the easiest. My idea was for a custom RSS feed system that would send out a chapter a day from one of the supported volumes (all scriptural works at this point). I call it ChaptersS.



This series of posts will cover my (in-)experience with Rails.

True Customer Service

Well, I received my Durango back from the garage last night. I had it in for several things: engine light, trailer brakes, and transfer case service. But those things aren't the point.

Several years ago we bought some discount service cards for the local Dodge dealership. The cards included a few oil changes and some discounts on service. Apparently the dealer received a few bucks from each card sold. Anyhow, the discount card supplier continued to sell the cards without authorization from the Dodge dealership. That put both the Dodge dealer and me in hard places. I paid money and expected a discount. The Dodge had a faithful customer in front of him expecting a discount but wasn't getting anything out of it. In an act of true customer service they honored the card. Wow!

They have made me a long term customer.

Computer Diagnoses Car - Follow Up

This is a follow up on a previous post regarding my car's engine light. My car's engine light came on again this morning. I guess it's time to take it back in to the dealer for service but that's OK because they do such a great job there.