Posted by admin on November 28, 2009 – 5:19 am
Filed under ruby
Tagged as code, crawler, domain, domain name, gem, http, ruby, spider, spidr, spidr gem, url, web crawler, web spider
Author: Kunto Aji
Site: http://www.railsmine.net
Summary: Ruby script to get all URLs from target site. You may need install Spidr gem first. This script is tested on Linux.
Usage: ruby filename.rb
#!/usb/bin/ruby
# _________ .__ .____ _______
# / _____/ ____ |__|_____ | | \ _ \ ____
# \_____ \ / \| \____ \| | / /_\ \ / ___\
# / \ | \ | |_> > |__\ \_/ \/ /_/ >
#/_______ /___| /__| __/|_______ \_____ /\___ /
# \/ \/ |__| \/ \//_____/
# http://www.railsmine.net
require 'rubygems'
require 'spidr'
i = 1
url_file = File.open('spider.txt', 'w')
Spidr.start_at('http://www.railsmine.net/') do |spider|
spider.every_url { |url|
puts "#{i}. #{url}"
if (url_file)
url_file.puts("#{i}. #{url}")
end
i = i + 1
}
end
url_file.close
puts "Done. All URLs has been saved to spider.txt"
Posted by admin on November 24, 2009 – 1:54 pm
Author: Isaacs
Site: http://gist.github.com/237575
Summary: print out the URLs requested system wide on the main network interface.
Posted by admin on November 21, 2009 – 4:46 pm
Filed under ruby
Tagged as code, email, gmail, imap, mail, message, ruby, ruby-gmail, server, snippet, unread
Author: Kunto Aji
Site: http://www.railsmine.net
Summary: Ruby script to get all unread messages from Gmail. Visit http://github.com/dcparker/ruby-gmail for more info about ruby-gmail gem. This script is tested on Linux.
Usage: ruby ruby-gmail.rb
#!/usb/bin/ruby
# _________ .__ .____ _______
# / _____/ ____ |__|_____ | | \ _ \ ____
# \_____ \ / \| \____ \| | / /_\ \ / ___\
# / \ | \ | |_> > |__\ \_/ \/ /_/ >
#/_______ /___| /__| __/|_______ \_____ /\___ /
# \/ \/ |__| \/ \//_____/
# http://www.railsmine.net
require 'rubygems'
require 'gmail'
username = "YOUR_USERNAME"
password = "YOUR_PASSWORD"
gmail = Gmail.new(username, email)
unread_messages = gmail.inbox.emails(:unread)
puts "All unread messages"
for mail in unread_messages
puts "----------------------------------"
puts "From: #{mail.message.from}"
puts "Subject: #{mail.message.subject}"
end
