Minasan, Watashiwa Wawan Desu...

Monday, February 14, 2011

Python code to grab KeywordDiscovery API data


If you use the KeywordDiscovery API, and Python, my pain is your gain. It took me a few hours to get this to work. You can grab it and go. Here's the function, written in my usual Python Pigdin. I don't recommend using it without a passing knowledge of Python, but that's up to you:

def kwdiscovery(username,password,phraselist):base64string = base64.encodestring('%s:%s' % (username, password))[:-1]authheader = "Basic %s" % base64stringapiurl = "http://api.keyworddiscovery.com/queries.php?queries="separator = "%0D%0A"counter = 1for phrase in phraselist:# make sure there's no funny characterstry: phrase.decode('ascii')except UnicodeDecodeError:continuephrase = phrase.replace(" ","+")phrase = phrase.replace("\n","")if (counter > 1):apiurl = apiurl + separator + phraseelse:apiurl = apiurl + phrasecounter = counter + 1apiurl = apiurl + "&empty=1"req = urllib2.Request(apiurl)req.add_header("Authorization", authheader)blah = urllib2.urlopen(req)# because sometimes, things just go wrongtry:result = ET.parse(blah)resultlist = []lst = result.findall("r")for item in lst:this = item.attrib["q"],item.attrib["m"]resultlist.append(this)except:this = "__one of the words in this request caused an error:",apiurlresultlist = [this]return resultlist

And here's how you'd use the function:

#!/usr/bin/pythonimport stringimport sysimport httplibimport urllib2from urllib2 import Request, urlopen, URLErrorimport xml.etree.ElementTree as ETimport base64f = open('longw.txt','r')g = open('words_countedlongtail.txt','w')words = f.readlines()username = "ENTER KEYWORDDISCOVERY USERNAME HERE"password = "ENTER KEYWORDDISCOVERY PASSWORD HERE"start = 0count = len(words)while (count > 0):count = count - 9end = start + 9a = words[start:end]print "sent ",aresultlist = kwdiscovery(username,password,a)for l in resultlist:q = str(l[0])m = str(l[1])line = q + "\t" + m + "\n"g.write(line)print "received ",linestart = endf.close()g.close()

Who knows, I might even create a web interface one of these days. In my spare time.



View the original article here

No comments: