Login
Forgot Password?

OR

Login with Google Login with Twitter Login with Facebook
  • Join
  • Profiles
  • Groups
  • SuicideGirls
  • Photos
  • Videos
  • Shop
Vital Stats

rook

Australia

Member Since 2004

Followers 87 Following 97

  • Everything
  • Photos
  • Video
  • Blogs
  • Groups
  • From Others

Sunday Jan 09, 2011

Jan 8, 2011
1
  • Facebook
  • Tweet
  • Email
For any Linux/Unix command-line commandos out there who may want to save their SuicideGirls blogs for posterity; the following command will scrape your pages and give you a text file of your content that you can then clean up and do with what you will.

curl -s "http://suicidegirls.com/members/username/blog/page[x-y]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > path/to/save/filename



Where: username is your SG user name (obviously); x is the latest page of your blog (1); and y is the last page of your blog (mine was 50).

So the actual command I used, as an example, was:

curl -s "http://suicidegirls.com/members/Rook/blog/page[1-50]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > /home/rook/Documents/SG_Blog.txt



Some caveats, however:

1 - You need to have curl and html2text installed on your system. Without curl you won't be able to scrape the pages. Without html2text you'll get an almost unreadable mess of html mark-up (I guess you could save that to a html file then open it in a browser and manually scrape it).

2 - Your blog needs to be public (although you may be able to get around that with curl's username and password switches. I don't know, I didn't try it).

3 - The output file will need a little manual clean-up, unless you like spending as much time fine-tuning a sed or awk command as it would take to do it the old fashioned way.

4 - This command does not grab the comments on your posts. You'll get a count of comments for each post, but there's no way to expand every post to expose your comments' source for the command to scrape. Sorry about that.

5 - If anyone knows a better/cleaner way of doing this, I'd love to hear from you!

6 - For you people in the iCult; you could probably get this work on your shiny new Macbook with a little tinkering to get the required programs in place.


SPOILERS! (Click to view)
Jesus H. Christ, I'm a nerd! It's amazing I've ever been laid.


VIEW 5 of 5 COMMENTS
pooky56:
Oh that is good. I do wish I was that nerdy. Might have to work on it..
Jan 29, 2011
conjure:
Ehhhhhhhhhh your blogs never make any sense (I never really give them a chance, granted, BUT it's because they always start with something that makes me completely lose concentration.) Glad to see that not much has changed around here wink
Feb 2, 2011

More Blogs

  • 02.01.09
    5

    Monday Feb 02, 2009

    Wait for it. 3... 2... 1...
  • 01.30.09
    2

    Friday Jan 30, 2009

    Read More
  • 01.26.09
    3

    Tuesday Jan 27, 2009

    It's about time I put up a blog post about the current happenings and…
  • 01.25.09
    0

    Monday Jan 26, 2009

    Hon Kevin Rudd MP Prime Minister of Australia Parliament House CANBER…
  • 01.16.09
    6

    Friday Jan 16, 2009

    Read More
  • 01.12.09
    6

    Monday Jan 12, 2009

    I think I'm addicted to change. I've realised that, like everyone …
  • 01.02.09
    8

    Saturday Jan 03, 2009

    Read More
  • 12.23.08
    7

    Wednesday Dec 24, 2008

    In this p…
  • 12.19.08
    3

    Friday Dec 19, 2008

    NOM NOM NOM... This is sound of me eating a pie made of humble crows…
  • 12.17.08
    6

    Thursday Dec 18, 2008

    Anybody care to tell me why I just got an email listing me as a MEMBE…

We at SuicideGirls have been celebrating alternative pin-up girls for:

23
years
10
months
24
days
  • 5,509,826 fans
  • 41,393 fans
  • 10,327,617 followers
  • 4,600 SuicideGirls
  • 1,114,303 followers
  • 14,949,527 photos
  • 321,315 followers
  • 61,464,887 comments
  • Join
  • Profiles
  • Groups
  • Photos
  • Videos
  • Shop
  • Help
  • About
  • Press
  • LIVE

Legal/Tos | DMCA | Privacy Policy | 18 U.S.C. 2257 Record-Keeping Requirements Compliance Statement | Contact Us | Vendo Payment Support
©SuicideGirls 2001-2025

Press enter to search
Fast Hi-res

Click here to join & see it all...

Crop your photo