Updated Ruby Code for Querying IpInfoDB

August 01, 2009

photo.jpg

This version adds two main features:

  • avoids barfing if the array size is zero when appending to the query string

  • sends the web service 25 queries at a time instead of one

  • speed: it’ll do about 3000 resolutions per minute now 

require 'rubygems'
require 'hpricot'
require 'open-uri'
ips = []
file = File.open('/Users/daniel/Development/newfile', 'w')
File.open('/Users/daniel/Development/ips3').each { |line| 
  ips.push line
}
while !ips.empty? do
  query = ""
  25.times do
    if !ips.empty?
      query += ips.pop.chomp + ","
    end
  end
query.chop!
doc = Hpricot(open("http://ipinfodb.com/ip_query2.php?ip=#{query}"))
(doc/'location').each do |el|
  ip = (el/'ip').inner_html
  country = (el/'countryname').inner_html
  state = (el/'regionname').inner_html
  city = (el/'city').inner_html
  puts "#{ip},#{country},#{state},#{city}"
  file.puts "#{ip},#{country},#{state},#{city}"
  file.flush()
end
end

[ Sorry for the mangled code; let me know if you want the clean version. ]

Next I’m going to try and pull the databases local and use hash tables like a co-worker did in .NET. He got the whole thing down to 8 seconds, regardless of how many items he had to query.

That’s the advantage of being local, but I think there’s also value in being more portable with the web call–especially when I can do a massive set of data in just a few minutes.

Anyway, I’m going to try the local version just for the fun of it. ::

Related posts: