YQL example -- get the direct URL for a file on SkyDrive

Yahoo Query Language (YQL) is an extremely useful and versatile free web service, API provided by Yahoo. YQL acts as a single, convenient point of entry for numerous other APIs. So, for example, you can use YQL to get inter-related data from a Bing search, a Google search, Twitter, and Flickr, through one simple call to YQL! You can also do sophisticated stuff like joining and filtering data sets (e.g. get YouTube videos and Flickr photos relating to the location where Twitter tweets were tweeted from) -- very cool stuff, perfect for mash-ups and hack-days, but beyond the scope of this article. The YQL infrastructure also does some important work for you, like cleaning and caching of data.

YQL is also capable of scraping HTML from web pages and parsing the data, all behind the scenes, allowing you to easily query just about any websites on the internet! In this simple demo, I'll show this feature in action, by using YQL to get the direct URL to a file on SkyDrive from the public web page for that file using pure client-side JavaScript.

(This is actually quite a useful example, because direct URLs to files on SkyDrive are temporary, so you always need to get the current URL before using it. I recently created a server-side library for getting permalinks from SkyDrive and a permalink redirector URL, and already hundreds of people are using it. YQL makes a client-side solution feasible, because YQL caches query results, and there is a good infrastructure behind it, yielding acceptable performance with client-side JavaScript, manifested as rapid second-pass execution at run-time.)

I was inspired to write this article during my Freestyle Friday hackday this afternoon, following a visit by Yahoo evangelist Christian Heilmann, with whom I've had some very useful contact during recent projects involving his organisation and mine.

YQL statement for getting a direct link to a file on SkyDrive

SELECT
    href
FROM
    html
WHERE
    url="http://cid-9115a524920cfd6f.office.live.com/self.aspx/.Public/Gaming/Halo%203/reach1196892-10261699.wmv"
    AND
    xpath="//a[@id='spPreviewLink']"

As you can see, YQL works a lot like SQL (the standard database query language) and is fairly human-readable and intuitive.

The YQL query is translated into a REST URL which can return data in the response as JSON or XML:

http://query.yahooapis.com/v1/public/yql?q=SELECT%20%0D%0A%09href%20%0D%0AFROM%20%0D%0A%09html%20%0D%0AWHERE%20%0D%0A%09url%3D%22http%3A%2F%2Fcid-9115a524920cfd6f.office.live.com%2Fself.aspx%2F.Public%2FGaming%2FHalo%25203%2Freach1196892-10261699.wmv%22%20%0D%0A%09AND%20%0D%0A%09xpath%3D%22%2F%2Fa%5B%40id%3D'spPreviewLink'%5D%22&format=json&diagnostics=true&callback=

Notice that I use a standard XPATH to select the HTML element of interest by its ID attribute. That's all there is to it, really!

I've set-up a YQL Query Alias called "GetSkyDriveFileDirectURLFromPublicPage" for this query, for convenience, under my YDN username "tim.acheson".

JavaScript code example using JQuery with JSONP

var skyDriveFilePublicPageURL = 'http://cid-9115a524920cfd6f.office.live.com/self.aspx/.Public/Gaming/Halo%203/reach1196892-10261699.wmv';
 
var jsonURITemplate = "http://query.yahooapis.com/v1/public/yql?q=SELECT%20%0D%0A%09href%20%0D%0AFROM%20%0D%0A%09html%20%0D%0AWHERE%20%0D%0A%09url%3D%22{skyDriveFilePublicPageURLEncoded}%22%20%0D%0A%09AND%20%0D%0A%09xpath%3D%22%2F%2Fa%5B%40id%3D'spPreviewLink'%5D%22&format=json&diagnostics=true&callback=GetSkyDriveFileDirectURLFromPublicPageCallback&jsoncallback=?";
 
var skyDriveFilePublicPageURLEncoded =  encodeURI(skyDriveFilePublicPageURL);
 
var jsonURI = jsonURITemplate.replace('{skyDriveFilePublicPageURLEncoded}', skyDriveFilePublicPageURLEncoded);
 
$.getJSON(
     jsonURI,
     function(jsonObject, statusText) {
           alert(jsonObject.query.results.a.href);
     }
);

Live working demo

LIVE DEMO: Get the direct URL for a file on SkyDrive from the URL of the file's public page

Click here to run the demo
Demo status: Ready

I hope you find this useful as an intro to YQL, which I think is one of the most useful and interesting web development tools that the web revolution has produced so far!

25 June 2010

Share the love:

Comments: 2

Add Comment


Nice article!

Tim Acheson (28 Jun 10, 11:12)

Thanks, my old coding comrade!

Tags:


  • Twitter
  • LinkedIn
  • Facebook
  • Windows Live / Messenger
  • Xbox Live
  • RSS
  • Email