0 Comments

(This is a blog post in progress. It will change over time. For now it is just an amalgamation of my current thoughts and experiences. I have not yet bothered to read dedicated literature regarding backups. I probably forgot some things I already know. You will probably find a better write-up on backups somewhere else. If you do, please send me your source :). I mean it.)


Why backups?

There are a myriad of reasons why you should back up your data:

  • disk failure
    • disk sectors not readable anymore
    • whole disk not readable due to driver error or similar
    • external physical force destroyed disk
  • accidental erasure
  • accidental change of data
  • loss of device with data
    • fire or other external hazards
    • theft
    • accidentally leaving device somewhere unrecoverable
  • loss of access to a service where your data resides
    • service looks you out of your account because of some dispute or because service is down for good
    • no access to password, email account, etc. to access service

It all depends on your personal preferences and how tragic a loss of data would be for you. This might be memories (photos, diaries), time investment (documents, programming code, access to password manager), …


What to back up?

In essence the answer is: everything you want to keep and cannot already restore via some method from somewhere else. Here are a few pointers:

  • documents
  • programming code
  • databases
  • hosted web pages
  • invoices
  • emails
  • calendars
  • bookmarks
  • software / games downloads (if bought)
  • software configurations (it can take quite some time to re-configure software so it behaves like with an old installation)
  • pictures
  • videos
  • audio files
  • data to access your password manager
  • license keys (for software)


When to back up?

As often as possible, but as rarely so that the process does not hinder your "normal" activities. It is a tradeoff between how much data are you willing to lose vs. how much time, money and energy do you want to invest in backups.


Where to back up to?

It depends on your personal taste for safety and data access. You can backup your data to an external storage (for instance an external SSD) – external referring to outside your device(s). You could keep that external storage at home, but it could become subject to a fire or to theft along with your devices. So keeping the backup storage in a separate location from the devices is usually good practice, for instance at a friends house.

You can backup your data to the cloud / personal server. Be sure to check your access to the external service / server regularly.


How to back up and restore?

This depends on what you want to achieve. Here are a few scenarios:

Disk failure:

  • restore last backup image of old disk on new disk
  • restore data from last backup on new disk (possibly install operating system and software anew)

Accidentally deleted / overwritten data since last backup:

  • restore single files from last backup
  • revert files to a previous state (if using repository-like backup)

Accidentally deleted / overwritten data before last backup:

  • restore files from an older backups than the accident
  • restore files from incremental backups from the time before accident happened
  • revert files to a state before the accident (if using repository-like backup)

Loss of access to service with data:

  • backup data via an export option in the service (choose the appropriate contents and format)
  • find a service (or local software) where you can import the backup you made from the service to which you lost access

Loss of device with data:

  • get new / used device, install operating system and software anew, restore data from last backup

Backup Tools:

Automate as much as you can – if not you will more prone to procrastinate your backups. There are many backup tools out there.

disk image: Clonezilla

data backup from one disk to another (external) disk: SyncBack

upload backups to AWS: JungleDisk (now CyberFortress)

Do not forget to backup your backup configurations if you want to re-setup those services faster next time.

Restoring:

Check intermittently that you can restore the data you backed up. It can be very frustrating if you rely on a backup and it is not restorable or does not contain the data you expect.

0 Comments

We use TeamCity on our servers to run unit test, performance tests and deployments. By now we rely heavily on the processes which we configured in TeamCity. In case we lose a server, we would like to have backup of all the configuration of TeamCity on that server.

To create daily backups we use TeamCitys REST API. There is an endpoint for Data Backup. I found a script written by Ivan Leonenko, which encapsulates the API call in a powershell script. If you want to then create a repeated Task Scheduler task, here is a step-by-step blog post on how to do that. Since we do all our backups with SyncBack, I created a repeated runner there which executes the powershell script. After that JungleDisk takes care of copying the backups to Amazon S3, where we would find our backup copy in case the server and its data is lost.