Blog Posts

State of Instacast Sync

I thought I give you a quick update about the state of podcast sync in Instacast. The current status quo is, that Instacast tries to synchronize its state with other devices via iCloud. We are using a technique known as document storage. Instacast simply saves its state into one single document that is pushed to other devices. Using this document Instacast on other devices can be updated accordingly. That’s the theory.

In practice there are a lot of pitfalls. One problem is that if you really want to sync data between devices somehow conflict free, the sync needs not only the current state but also information about what has happend in the past. This is not easy to do with iCloud. iCloud does not have a syncing service per se. Everything is stored in files. If you just change one property on one object, a new file has to be written. You can consolidate multiple changes into one file, but that increases syncing delays. Also iCloud has limitations when it comes to the number of files it can reliably store. On top iCloud seems to have a problem with connection reliability. It is very possible that data can not be downloaded or that the file transfer process hangs.

All this does not make it a joy to develop for iCloud. And that’s why so many 3rd party developers have problems with their iCloud integration.

I thought about this problem very hard for a long time. I would like to have a reliable syncing solution that works instantanously, is fast and doesn’t suffer from erratic glitches. I’ve come to the conclusion that these goals are not achievable with iCloud at the moment and may not be for the forseeable future. So what’s the alternative?

One alternative would be to develop my own syncing solution and roll out my own server infrastructure. This would have the advantage of having total control over everything, meaning if something breaks I would have the power to fix it no matter what. The downside is that I have to put a lot of effort in developing a syncing algorithm that is reliable and scalable at the same time. Believe me when I say, that this is the holy grail of software engineering right now. The other downside would be that I would have to manage my own server infrastucture, including running the service, running backups, maintaining hardware issues etc. This would consume a lot of time that I could spend a lot better developing apps.

The other alternative would be to find another server provider specialized in data syncing across devices. The advantage would be that I don’t need to maintain server reliability and security and that I don’t need to reinvent the syncing wheel. The disadvantage of course is to give up control. Luckily I found a service provider that looks very promising. First tests have shown that their sync works amazingly well and is very fast and quite reliable. I am talking about Simperium, the guys behind Simplenote. They took their syncing experience and created an awesome service. I am very impressed and at the same time very disappointed by Apple that they are not able to achieve what a small team of four guys did. The service is powered by Amazon's AWS, so the servers are really reliable and the transfer speed is awesome.

Unfortunatly both alternatives are not available for free. Simperium needs a couple of cents per user per month that can add up to serious money real quick given a certain number of users. It’s clear that I can not switch the sync to something better without the requirement of a subscription fee. However I think it’s really worth it and that in the end everybody would happily pay some small money in exchange of something better.

I was also ask often if I couldn’t simply use Dropbox. Dropbox while being and awesome server for file storage is not the best solution to sync a database. To some degree it would also suffer from performance issues like iCloud. I considered it for a while, but gave up on it because it lacks some technical features to get a decent sync to work.

I haven’t decided yet what to do. I am currently testing both alternatives. Will let you know when I make progress. If you have any comments, please let me know. You can follow me on Twitter.

August 6th, 2012 • Permalink

More articles about Blog Posts

09/16/2014 - Updates on Instacast 5
06/27/2014 - Instacast Cloud Memberships
06/06/2014 - Instacast 4.6 and Instacast Cloud
03/26/2014 - Sharebox adds Drag & Drop sharing to the Dropbox menubar icon
01/22/2014 - Instacast 4.2 adds News Mode
01/12/2014 - Hello 2014
11/07/2013 - Instacast Available on the Mac App Store
10/23/2013 - New Payment Option: Bitcoin
10/21/2013 - Spotlight... The Psych Files
10/02/2013 - Instacast Cloud Sync Outage
09/21/2013 - Quick Tips for Instacast 4