rook

For any Linux/Unix command-line commandos out there who may want to save their SuicideGirls blogs for posterity; the following command will scrape your pages and give you a text file of your content that you can then clean up and do with what you will.

Where: username is your SG user name (obviously); x is the latest page of your blog (1); and y is the last page of your blog (mine was 50).

So the actual command I used, as an example, was:

Some caveats, however:

1 - You need to have curl and html2text installed on your system. Without curl you won't be able to scrape the pages. Without html2text you'll get an almost unreadable mess of html mark-up (I guess you could save that to a html file then open it in a browser and manually scrape it).

2 - Your blog needs to be public (although you may be able to get around that with curl's username and password switches. I don't know, I didn't try it).

3 - The output file will need a little manual clean-up, unless you like spending as much time fine-tuning a sed or awk command as it would take to do it the old fashioned way.

4 - This command does not grab the comments on your posts. You'll get a count of comments for each post, but there's no way to expand every post to expose your comments' source for the command to scrape. Sorry about that.

5 - If anyone knows a better/cleaner way of doing this, I'd love to hear from you!

6 - For you people in the iCult; you could probably get this work on your shiny new Macbook with a little tinkering to get the required programs in place.