Skip to main content
September 29, 2011

Third Party Data Woes

In this post, Bee talks about taking precautionary steps when using third party data…

Bee Thao

No, this isn’t third party dating woes; however, this does pertain to relationships. One of the cool things you can do on a website is to integrate information from other sites. You can embed YouTube ® videos on your site, pull in your blog posts from Blogger, or allow your visitors to register for your MailChimp e-newsletter right from your website. Majors services like Facebook, Twitter, YouTube® all have their own data feed or application programming interface (API) that you can use to integrate their data seamlessly on your website.
 
This definitely allows for a more interactive, interesting and content- rich site. What happens though, if one of those services no longer work or is offline for some reason? Depending on how you’re integrating the data, it may or may not affect how your website is functioning.
 
I’ve had three incidents this year in which using third party data caused sites using them to go down. In one incident, I was scraping the data. Basically, retrieving the HTML off the site and plopping into the site what I needed. When that source site suddenly went offline, it also took down the site. In the last two incidents, two very popular services, Blogger and Vimeo went offline for a short while and also took down sites using their data feeds.
 
These are relationships that work when they’re all online, but can cause distress when they go offline. A lesson I learned from these incidents is that you shouldn’t always count on third-party sites to be online all the time, even if they are popular. An alternative should be in place in case it does occur.
 
There are some precautions you can take to make sure that your relationship with third-party data isn’t going to affect your site when they no longer work. In the first incident, I wrote a script to make a backup of the current data feed each day so that there is always a fresh copy on the server. When the source site is inaccessible, the backup is used. For Vimeo, I created a ‘temporarily unavailable’ image that is displayed when it is detected that Vimeo’s feed is down. That way a user will know there are some technical issues at the moment.
 
Although working with third-party data is a great way to expand and enrich your site, you should be aware of and plan for outages. If you’ve experienced any of these issues, I’d love to hear your stories and solutions.