Backing up and restoring

Overview

The ThingPark Enterprise backup process allows to create and archive a snapshot of all platform data. Timeseries data are excluded from the backup : devices and base stations statistics and wlogger packets are not backed up. Backup and restore are manual operations triggered through Kubernetes API exec commands. A scheduled backup can also be configured.

caution

Archive could only be restored on the same ThingPark Enterprise version.
Configuration customization backup is not part of this process
If the backup is restored after a Kubernetes control plan disaster, a fresh install must be done first.

Backing Up

Manually backing up data

A manual backup can be triggered by running backup script in the tp-backup-controller deployment context.

Run backup script using kubernetes api exec endpoint:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- backup

Backup is pushed to blob storage:

localhost ok: {
    "changed": false,
    "msg": "New backup thingpark-enterprise-backup-20220120091609 successfully pushed to remote storage"
}

Scheduling backups

Scheduled backup can be enabled after initial deployment by updating configuration of the Helm thingpark-enterprise release.

Add a schedule key in backup block with appropriate cron expression:
```
backup:
schedule: "30 2 * * *"
```

Upgrade the chart release:

helm upgrade -i tpe -n $NAMESPACE \
  actility/thingpark-enterprise --version $THINGPARK_ENTERPRISE_VERSION \
  -f $CONFIG_REPO_BASEURL/configs/$HOSTING/values-$SEGMENT-segment.yaml \
  -f custom-values.yaml

Backups retention should be set using cloud storage features:

Restoring data

The restore action must be done on a TPE instance already installed and configured.

Identify a backup that you want to restore by listing available ones:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- list-backups

Validate that backup chart, appVersion and releaseNamespace fields match with redeployed version:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- get-backup-metadatas -e backup_name=%backup name%

Trigger the data restoration (command will ask for confirmation):

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- restore -e backup_name=%backup name%

Rollback

This procedure applies to a rollback for any reason after an upgrade:

Identify latest backup automatically triggered during upgrade:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- list-backups

Rollback to the previous Helm revision:
```
helm rollback -n $NAMESPACE tpe
```

List available backups to retrieve the last one done at pre upgrade step:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- list-backups

After all Deployment/Statefulset back to an Available state, start the restoration procedure using identified backup:

$ kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- restore -e backup_name=<backup name>
Executing playbook restore.yml

- Start restoring a data backup on hosts: localhost -
Gathering Facts...
localhost ok
Check parameters...
localhost ok: {
    "changed": false,
    "msg": "thingpark-enterprise-backup-20220120093259 backup will be use to restore"
}
WARNING: Destructive Operation !!!...
[WARNING: Destructive Operation !!!]
This Operation will restore a previous state of your TPE instance.
Are you sure you want to delete current datas? (yes/no)

note

If you have already retried helm upgrade, the rollback can be done to a previous Helm Release Revision:

Identify the release revision you want to rollback:

$ helm history -n $NAMESPACE tpe
REVISION    UPDATED                     STATUS      CHART                               APP VERSION DESCRIPTION
...
8           Thu Jan 20 10:24:54 2022    superseded  thingpark-enterprise-1.0.2  7.1.0       Upgrade complete
...

Trigger the Helm Revision rollback:

helm rollback -n $NAMESPACE tpe <revision>

Rollback a thingpark-enterprise configuration revision

This procedure allows to restore a previous Helm revision with data restoration:

Identify the release revision you want to rollback to and take a note of APP VERSION :

$ helm history -n $NAMESPACE tpe
REVISION    UPDATED                     STATUS      CHART                               APP VERSION DESCRIPTION
...
8           Thu Jan 20 10:24:54 2022    superseded  thingpark-enterprise-1.0.2  7.1.0       Upgrade complete
...

List available backups to retrieve the one that you want to restore

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- list-backups

Validate that backup appVersion stored in its metadatas match with your ThingPark X.Y.Z Helm revision APP VERSION:

$ kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- get-backup-metadatas -e backup_name=<backup name>
...

Backup metadatas:...
localhost ok: {
    "changed": false,
    "msg": [
        "helm:",
        "    appVersion: 7.1.0",
        "    chart: thingpark-enterprise-1.0.2",
        "    releaseName: tpe",
        "    releaseNamespace: thingpark",
        "    revision: '7'",
        "id: thingpark-enterprise-backup-20220120093259",
        "tm: '2022-01-20T09:32:59Z'",
        "type: on-demand"
    ]
}
...

Trigger the Helm revision rollback:

helm rollback -n $NAMESPACE tpe <revision>

After all Deployment/Statefulset back to an Available state, start the restoration procedure using identified backup:

$ kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- restore -e backup_name=<backup name>
Executing playbook restore.yml

- Start restoring a data backup on hosts: localhost -
Gathering Facts...
localhost ok
Check parameters...
localhost ok: {
    "changed": false,
    "msg": "thingpark-enterprise-backup-20220120093259 backup will be use to restore"
}
WARNING: Destructive Operation !!!...
[WARNING: Destructive Operation !!!]
This Operation will restore a previous state of your TPE instance.
Are you sure you want to delete current datas? (yes/no)

Backing up and restoring

Overview​

Backing Up​

Manually backing up data​

Scheduling backups​

Restoring data​

Rollback​

Rollback a thingpark-enterprise configuration revision​