Login
Forgot Password?

OR

Login with Google Login with Twitter Login with Facebook
  • Join
  • Profiles
  • Groups
  • SuicideGirls
  • Photos
  • Videos
  • Shop
Vital Stats

rook

Australia

Member Since 2004

Followers 87 Following 97

  • Everything
  • Photos
  • Video
  • Blogs
  • Groups
  • From Others

Sunday Jan 09, 2011

Jan 8, 2011
1
  • Facebook
  • Tweet
  • Email
For any Linux/Unix command-line commandos out there who may want to save their SuicideGirls blogs for posterity; the following command will scrape your pages and give you a text file of your content that you can then clean up and do with what you will.

curl -s "http://suicidegirls.com/members/username/blog/page[x-y]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > path/to/save/filename



Where: username is your SG user name (obviously); x is the latest page of your blog (1); and y is the last page of your blog (mine was 50).

So the actual command I used, as an example, was:

curl -s "http://suicidegirls.com/members/Rook/blog/page[1-50]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > /home/rook/Documents/SG_Blog.txt



Some caveats, however:

1 - You need to have curl and html2text installed on your system. Without curl you won't be able to scrape the pages. Without html2text you'll get an almost unreadable mess of html mark-up (I guess you could save that to a html file then open it in a browser and manually scrape it).

2 - Your blog needs to be public (although you may be able to get around that with curl's username and password switches. I don't know, I didn't try it).

3 - The output file will need a little manual clean-up, unless you like spending as much time fine-tuning a sed or awk command as it would take to do it the old fashioned way.

4 - This command does not grab the comments on your posts. You'll get a count of comments for each post, but there's no way to expand every post to expose your comments' source for the command to scrape. Sorry about that.

5 - If anyone knows a better/cleaner way of doing this, I'd love to hear from you!

6 - For you people in the iCult; you could probably get this work on your shiny new Macbook with a little tinkering to get the required programs in place.


SPOILERS! (Click to view)
Jesus H. Christ, I'm a nerd! It's amazing I've ever been laid.


VIEW 5 of 5 COMMENTS
pooky56:
Oh that is good. I do wish I was that nerdy. Might have to work on it..
Jan 29, 2011
conjure:
Ehhhhhhhhhh your blogs never make any sense (I never really give them a chance, granted, BUT it's because they always start with something that makes me completely lose concentration.) Glad to see that not much has changed around here wink
Feb 2, 2011

More Blogs

  • 03.26.08
    6

    Wednesday Mar 26, 2008

    I found my new tattoo...but where should I put it‽ SPOILERS! …
  • 03.16.08
    2

    Monday Mar 17, 2008

    Show the world where you go and what you see! Yellow Arrow Project
  • 03.10.08
    0

    Tuesday Mar 11, 2008

    Pride...Check Envy...Check Gluttony...Check Greed...Check Lust...Chec…
  • 03.04.08
    4

    Wednesday Mar 05, 2008

    You know what? Fuck Schrdinger and finger fuck his fucking cat. Fucke…
  • 03.01.08
    2

    Saturday Mar 01, 2008

    The room smells like sex and cigarettes. The bed looks like we fought…
  • 02.22.08
    2

    Friday Feb 22, 2008

    While I'm not actually depressed at the moment, I just had a sobering…
  • 02.19.08
    4

    Tuesday Feb 19, 2008

    The times, they are a-changin'... My life is set to metamorphise qui…
  • 02.18.08
    4

    Monday Feb 18, 2008

    Hola! I thought it was about time I got on here and updated my life'…
  • 02.07.08
    5

    Friday Feb 08, 2008

    It's time for a new post but I have nothing to say. Some aspiring wr…
  • 02.01.08
    2

    Saturday Feb 02, 2008

    I'm loving my weekends dearly lately. Don't get me wrong, I've always…

We at SuicideGirls have been celebrating alternative pin-up girls for:

23
years
11
months
22
days
  • 5,509,826 fans
  • 41,393 fans
  • 10,327,617 followers
  • 4,608 SuicideGirls
  • 0 followers
  • 14,964,548 photos
  • 321,315 followers
  • 61,502,302 comments
  • Join
  • Profiles
  • Groups
  • Photos
  • Videos
  • Shop
  • Help
  • About
  • Press
  • LIVE

Legal/Tos | DMCA | Privacy Policy | 18 U.S.C. 2257 Record-Keeping Requirements Compliance Statement | Contact Us | Vendo Payment Support
©SuicideGirls 2001-2025

Press enter to search
Fast Hi-res

Click here to join & see it all...

Crop your photo