Login
Forgot Password?

OR

Login with Google Login with Twitter Login with Facebook
  • Join
  • Profiles
  • Groups
  • SuicideGirls
  • Photos
  • Videos
  • Shop
Vital Stats

rook

Australia

Member Since 2004

Followers 87 Following 97

  • Everything
  • Photos
  • Video
  • Blogs
  • Groups
  • From Others

Sunday Jan 09, 2011

Jan 8, 2011
1
  • Facebook
  • Tweet
  • Email
For any Linux/Unix command-line commandos out there who may want to save their SuicideGirls blogs for posterity; the following command will scrape your pages and give you a text file of your content that you can then clean up and do with what you will.

curl -s "http://suicidegirls.com/members/username/blog/page[x-y]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > path/to/save/filename



Where: username is your SG user name (obviously); x is the latest page of your blog (1); and y is the last page of your blog (mine was 50).

So the actual command I used, as an example, was:

curl -s "http://suicidegirls.com/members/Rook/blog/page[1-50]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > /home/rook/Documents/SG_Blog.txt



Some caveats, however:

1 - You need to have curl and html2text installed on your system. Without curl you won't be able to scrape the pages. Without html2text you'll get an almost unreadable mess of html mark-up (I guess you could save that to a html file then open it in a browser and manually scrape it).

2 - Your blog needs to be public (although you may be able to get around that with curl's username and password switches. I don't know, I didn't try it).

3 - The output file will need a little manual clean-up, unless you like spending as much time fine-tuning a sed or awk command as it would take to do it the old fashioned way.

4 - This command does not grab the comments on your posts. You'll get a count of comments for each post, but there's no way to expand every post to expose your comments' source for the command to scrape. Sorry about that.

5 - If anyone knows a better/cleaner way of doing this, I'd love to hear from you!

6 - For you people in the iCult; you could probably get this work on your shiny new Macbook with a little tinkering to get the required programs in place.


SPOILERS! (Click to view)
Jesus H. Christ, I'm a nerd! It's amazing I've ever been laid.


VIEW 5 of 5 COMMENTS
pooky56:
Oh that is good. I do wish I was that nerdy. Might have to work on it..
Jan 29, 2011
conjure:
Ehhhhhhhhhh your blogs never make any sense (I never really give them a chance, granted, BUT it's because they always start with something that makes me completely lose concentration.) Glad to see that not much has changed around here wink
Feb 2, 2011

More Blogs

  • 05.02.09
    10

    Sunday May 03, 2009

    Random question for the girls... SPOILERS! (Click to view)Is a guy…
  • 04.27.09
    3

    Tuesday Apr 28, 2009

    Thanks for the good thoughts from those who commented on my last post…
  • 04.26.09
    3

    Monday Apr 27, 2009

    Wow, this post is going to be quite a contrast with the good news I p…
  • 04.15.09
    4

    Thursday Apr 16, 2009

    I found $1100 today. I logged into my bank account to see if I'd…
  • 04.07.09
    2

    Tuesday Apr 07, 2009

    I'm now a proud new member of the SG Geezers group. It feels like hom…
  • 04.06.09
    2

    Monday Apr 06, 2009

    Things are both up and very much down right now. The 'up' is that I …
  • 04.02.09
    2

    Friday Apr 03, 2009

    Right now, right at this very second, I'm thinking I've missed the po…
  • 03.29.09
    3

    Sunday Mar 29, 2009

    I don't usually bother posting Twitter stuff on here, but y'all NEED …
  • 03.26.09
    2

    Thursday Mar 26, 2009

    Holy shit. Against my cynical nature and despite my hardened heart…
  • 03.20.09
    5

    Friday Mar 20, 2009

    Holy Jesus, Joseph and Mary, Mother of Fuck!! I just viewed a set so…

We at SuicideGirls have been celebrating alternative pin-up girls for:

23
years
10
months
22
days
  • 5,509,826 fans
  • 41,393 fans
  • 10,327,617 followers
  • 4,600 SuicideGirls
  • 1,114,898 followers
  • 14,948,517 photos
  • 321,315 followers
  • 61,461,889 comments
  • Join
  • Profiles
  • Groups
  • Photos
  • Videos
  • Shop
  • Help
  • About
  • Press
  • LIVE

Legal/Tos | DMCA | Privacy Policy | 18 U.S.C. 2257 Record-Keeping Requirements Compliance Statement | Contact Us | Vendo Payment Support
©SuicideGirls 2001-2025

Press enter to search
Fast Hi-res

Click here to join & see it all...

Crop your photo