Minasan, Watashiwa Wawan Desu...

NurCell Movies

Monday, February 14, 2011

Python code to grab KeywordDiscovery API data


If you use the KeywordDiscovery API, and Python, my pain is your gain. It took me a few hours to get this to work. You can grab it and go. Here's the function, written in my usual Python Pigdin. I don't recommend using it without a passing knowledge of Python, but that's up to you:

def kwdiscovery(username,password,phraselist):base64string = base64.encodestring('%s:%s' % (username, password))[:-1]authheader = "Basic %s" % base64stringapiurl = "http://api.keyworddiscovery.com/queries.php?queries="separator = "%0D%0A"counter = 1for phrase in phraselist:# make sure there's no funny characterstry: phrase.decode('ascii')except UnicodeDecodeError:continuephrase = phrase.replace(" ","+")phrase = phrase.replace("\n","")if (counter > 1):apiurl = apiurl + separator + phraseelse:apiurl = apiurl + phrasecounter = counter + 1apiurl = apiurl + "&empty=1"req = urllib2.Request(apiurl)req.add_header("Authorization", authheader)blah = urllib2.urlopen(req)# because sometimes, things just go wrongtry:result = ET.parse(blah)resultlist = []lst = result.findall("r")for item in lst:this = item.attrib["q"],item.attrib["m"]resultlist.append(this)except:this = "__one of the words in this request caused an error:",apiurlresultlist = [this]return resultlist

And here's how you'd use the function:

#!/usr/bin/pythonimport stringimport sysimport httplibimport urllib2from urllib2 import Request, urlopen, URLErrorimport xml.etree.ElementTree as ETimport base64f = open('longw.txt','r')g = open('words_countedlongtail.txt','w')words = f.readlines()username = "ENTER KEYWORDDISCOVERY USERNAME HERE"password = "ENTER KEYWORDDISCOVERY PASSWORD HERE"start = 0count = len(words)while (count > 0):count = count - 9end = start + 9a = words[start:end]print "sent ",aresultlist = kwdiscovery(username,password,a)for l in resultlist:q = str(l[0])m = str(l[1])line = q + "\t" + m + "\n"g.write(line)print "received ",linestart = endf.close()g.close()

Who knows, I might even create a web interface one of these days. In my spare time.



View the original article here

No comments: