YQL example -- get the direct URL for a file on SkyDrive
Yahoo Query Language (YQL) is an extremely useful and versatile free web service, API provided by Yahoo. YQL acts as a single, convenient point of entry for numerous other APIs. So, for example, you can use YQL to get inter-related data from a Bing search, a Google search, Twitter, and Flickr, through one simple call to YQL! You can also do sophisticated stuff like joining and filtering data sets (e.g. get YouTube videos and Flickr photos relating to the location where Twitter tweets were tweeted from) -- very cool stuff, perfect for mash-ups and hack-days, but beyond the scope of this article. The YQL infrastructure also does some important work for you, like cleaning and caching of data.
YQL is also capable of scraping HTML from web pages and parsing the data, all behind the scenes, allowing you to easily query just about any websites on the internet! In this simple demo, I'll show this feature in action, by using YQL to get the direct URL to a file on SkyDrive from the public web page for that file using pure client-side JavaScript.
(This is actually quite a useful example, because direct URLs to files on SkyDrive are temporary, so you always need to get the current URL before using it. I recently created a server-side library for getting permalinks from SkyDrive and a permalink redirector URL, and already hundreds of people are using it. YQL makes a client-side solution feasible, because YQL caches query results, and there is a good infrastructure behind it, yielding acceptable performance with client-side JavaScript, manifested as rapid second-pass execution at run-time.)
I was inspired to write this article during my Freestyle Friday hackday this afternoon, following a visit by Yahoo evangelist Christian Heilmann, with whom I've had some very useful contact during recent projects involving his organisation and mine.
YQL statement for getting a direct link to a file on SkyDrive
href
FROM
html
WHERE
url="http://cid-9115a524920cfd6f.office.live.com/self.aspx/.Public/Gaming/Halo%203/reach1196892-10261699.wmv"
AND
xpath="//a[@id='spPreviewLink']"
As you can see, YQL works a lot like SQL (the standard database query language) and is fairly human-readable and intuitive.
The YQL query is translated into a REST URL which can return data in the response as JSON or XML:
Notice that I use a standard XPATH to select the HTML element of interest by its ID attribute. That's all there is to it, really!
I've set-up a YQL Query Alias called "GetSkyDriveFileDirectURLFromPublicPage" for this query, for convenience, under my YDN username "tim.acheson".
JavaScript code example using JQuery with JSONP
var jsonURITemplate = "http://query.yahooapis.com/v1/public/yql?q=SELECT%20%0D%0A%09href%20%0D%0AFROM%20%0D%0A%09html%20%0D%0AWHERE%20%0D%0A%09url%3D%22{skyDriveFilePublicPageURLEncoded}%22%20%0D%0A%09AND%20%0D%0A%09xpath%3D%22%2F%2Fa%5B%40id%3D'spPreviewLink'%5D%22&format=json&diagnostics=true&callback=GetSkyDriveFileDirectURLFromPublicPageCallback&jsoncallback=?";
var skyDriveFilePublicPageURLEncoded = encodeURI(skyDriveFilePublicPageURL);
var jsonURI = jsonURITemplate.replace('{skyDriveFilePublicPageURLEncoded}', skyDriveFilePublicPageURLEncoded);
$.getJSON(
jsonURI,
function(jsonObject, statusText) {
alert(jsonObject.query.results.a.href);
}
);
Live working demo
LIVE DEMO: Get the direct URL for a file on SkyDrive from the URL of the file's public page
Click here to run the demo
Demo status: Ready
I hope you find this useful as an intro to YQL, which I think is one of the most useful and interesting web development tools that the web revolution has produced so far!
25 June 2010
Tags: yql yahoo query language skydrive json jquery jsonp scrape html api apis mashup mash-up hack day freestyle friday demo example sample code javascript yahoo rest restful web service
Comments: 2
Add Comment
Nice article!
Thanks, my old coding comrade!