Login
Forgot Password?

OR

Login with Google Login with Twitter Login with Facebook
  • Join
  • Profiles
  • Groups
  • SuicideGirls
  • Photos
  • Videos
  • Shop
Vital Stats

rook

Australia

Member Since 2004

Followers 87 Following 97

  • Everything
  • Photos
  • Video
  • Blogs
  • Groups
  • From Others

Sunday Jan 09, 2011

Jan 8, 2011
1
  • Facebook
  • Tweet
  • Email
For any Linux/Unix command-line commandos out there who may want to save their SuicideGirls blogs for posterity; the following command will scrape your pages and give you a text file of your content that you can then clean up and do with what you will.

curl -s "http://suicidegirls.com/members/username/blog/page[x-y]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > path/to/save/filename



Where: username is your SG user name (obviously); x is the latest page of your blog (1); and y is the last page of your blog (mine was 50).

So the actual command I used, as an example, was:

curl -s "http://suicidegirls.com/members/Rook/blog/page[1-50]/" | sed -n '/timestamp/,/blogContentEdit/p' | html2text > /home/rook/Documents/SG_Blog.txt



Some caveats, however:

1 - You need to have curl and html2text installed on your system. Without curl you won't be able to scrape the pages. Without html2text you'll get an almost unreadable mess of html mark-up (I guess you could save that to a html file then open it in a browser and manually scrape it).

2 - Your blog needs to be public (although you may be able to get around that with curl's username and password switches. I don't know, I didn't try it).

3 - The output file will need a little manual clean-up, unless you like spending as much time fine-tuning a sed or awk command as it would take to do it the old fashioned way.

4 - This command does not grab the comments on your posts. You'll get a count of comments for each post, but there's no way to expand every post to expose your comments' source for the command to scrape. Sorry about that.

5 - If anyone knows a better/cleaner way of doing this, I'd love to hear from you!

6 - For you people in the iCult; you could probably get this work on your shiny new Macbook with a little tinkering to get the required programs in place.


SPOILERS! (Click to view)
Jesus H. Christ, I'm a nerd! It's amazing I've ever been laid.


VIEW 5 of 5 COMMENTS
pooky56:
Oh that is good. I do wish I was that nerdy. Might have to work on it..
Jan 29, 2011
conjure:
Ehhhhhhhhhh your blogs never make any sense (I never really give them a chance, granted, BUT it's because they always start with something that makes me completely lose concentration.) Glad to see that not much has changed around here wink
Feb 2, 2011

More Blogs

  • 05.16.08
    0

    Saturday May 17, 2008

    I think I killed my laptop. Damned GRUB! Damned MBR! Damned sixth ins…
  • 05.14.08
    6

    Thursday May 15, 2008

    Ma'fucker took my name! Kinda...
  • 05.12.08
    3

    Tuesday May 13, 2008

    Follow my Linux (and other techno-centric) exploits over here. Feel …
  • 05.05.08
    4

    Tuesday May 06, 2008

    I'm studying a unit entitled 'Virtual Cultures' this semester. As par…
  • 04.28.08
    7

    Tuesday Apr 29, 2008

    I know ya'll won't much care but I'm reserving tonight just for leisu…
  • 04.25.08
    0

    Friday Apr 25, 2008

    I downloaded this last week and watched it last night: Ben X The …
  • 04.22.08
    2

    Tuesday Apr 22, 2008

    Time to bitch about my work! I HATE most of the people I have to wor…
  • 04.13.08
    13

    Sunday Apr 13, 2008

    Read More
  • 04.08.08
    4

    Wednesday Apr 09, 2008

    I can't imagine what I used to blog so regularly about. Either my li…
  • 04.01.08
    7

    Tuesday Apr 01, 2008

    I found this in the Futurism group. It's a little long and off-putti…

We at SuicideGirls have been celebrating alternative pin-up girls for:

23
years
11
months
15
days
  • 5,509,826 fans
  • 41,393 fans
  • 10,327,617 followers
  • 4,608 SuicideGirls
  • 1,112,986 followers
  • 14,960,572 photos
  • 321,315 followers
  • 61,494,158 comments
  • Join
  • Profiles
  • Groups
  • Photos
  • Videos
  • Shop
  • Help
  • About
  • Press
  • LIVE

Legal/Tos | DMCA | Privacy Policy | 18 U.S.C. 2257 Record-Keeping Requirements Compliance Statement | Contact Us | Vendo Payment Support
©SuicideGirls 2001-2025

Press enter to search
Fast Hi-res

Click here to join & see it all...

Crop your photo