Backing up and restoring

Overview

The ThingPark Enterprise backup process allows to create and archive a snapshot of all platform data. Timeseries data are excluded from the backup : devices and base stations statistics and wlogger packets are not backed up. Backup and restore are manual operations triggered through Kubernetes API exec commands. A scheduled backup can also be configured.

caution

Archive could only be restored on the same ThingPark Enterprise version.
Configuration customization backup is not part of this process
If the backup is restored after a Kubernetes control plan disaster, a fresh install must be done first.

Backing Up

Manually backing up data

A manual backup can be triggered by running backup script in the tp-backup-controller deployment context.

Run backup script using kubernetes api exec endpoint

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o backup

Successful backup push archive to blob storage

2024-07-25T09:16:57.205Z	INFO	tp-backup/main.go:52	Backup succeed

Scheduling backups

Scheduled backup can be enabled after initial deployment by updating configuration of the Helm thingpark-enterprise release.

Add a schedule key in backup block with appropriate cron expression:
```
backup:
schedule: "30 2 * * *"
```

Upgrade the chart release:

helm upgrade -i tpe -n $NAMESPACE \
  actility/thingpark-enterprise --version $THINGPARK_ENTERPRISE_VERSION \
  -f $CONFIG_REPO_BASEURL/configs/$HOSTING/values-$SEGMENT-segment.yaml \
  -f custom-values.yaml

Backups retention should be set using cloud storage features:

Restoring data

The restore action must be done on a TPE instance already installed and configured.

Identify a backup that you want to restore by listing available ones:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o list

Trigger the data restoration (command will ask for confirmation):

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o restore -b %backup name%

Rollback

This procedure applies to a rollback for any reason after an upgrade:

Identify latest backup automatically triggered during upgrade:

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o list

Rollback to the previous Helm revision:
```
helm rollback -n $NAMESPACE tpe
```

After all Deployment/Statefulset back to an Available state, start the restoration procedure using identified backup:

$ kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o restore -b <backup name>
   2024-07-25T09:20:20.918Z	INFO	logger/logger.go:52	Logger construction succeeded
   2024-07-25T09:20:20.920Z	INFO	tp-backup/main.go:39	Initialize controller with Loaded Configuration
   2024-07-25T09:20:20.921Z	INFO	storage/azureblob.go:60	Azure Blob storage initialized
   2024-07-25T09:20:20.937Z	INFO	datasource/mariadb.go:49	Wireless datasource Initialized
   2024-07-25T09:20:20.988Z	INFO	datasource/mongodb.go:82	Wireless datasource Initialized
   2024/07/25 09:20:20 connected to 10.240.32.71:2181
   2024/07/25 09:20:21 authenticated: id=144132615539458050, timeout=4000
   2024/07/25 09:20:21 re-submitting `0` credentials after reconnect
   2024-07-25T09:20:22.090Z	INFO	datasource/core.go:80	Core datasource Initialized
   2024-07-25T09:20:22.091Z	INFO	datasource/kubernetes.go:68	Kubernetes datasource Initialized
   WARNING: Destructive Operation !!!
   This Operation will restore a previous state of your TPE instance.
   Are you sure you want to delete current datas? (yes/no)

note

If you have already retried helm upgrade, the rollback can be done to a previous Helm Release Revision:

Identify the release revision you want to rollback:

$ helm history -n $NAMESPACE tpe
REVISION	UPDATED                 	STATUS    	CHART                           	APP VERSION	DESCRIPTION
...
8       	Thu Jan 20 10:24:54 2022	superseded	thingpark-enterprise-1.0.2	7.1.0      	Upgrade complete
...

Trigger the Helm Revision rollback:

helm rollback -n $NAMESPACE tpe <revision>

Rollback a thingpark-enterprise configuration revision

This procedure allows to restore a previous Helm revision with data restoration:

Identify the release revision you want to rollback to and take a note of APP VERSION :

$ helm history -n $NAMESPACE tpe
REVISION	UPDATED                 	STATUS    	CHART                           	APP VERSION	DESCRIPTION
...
8       	Thu Jan 20 10:24:54 2022	superseded	thingpark-enterprise-1.0.2	7.1.0      	Upgrade complete
...

List available backups to retrieve the one that you want to restore

kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o list

Trigger the Helm revision rollback:

helm rollback -n $NAMESPACE tpe <revision>

After all Deployment/Statefulset back to an Available state, start the restoration procedure using identified backup:

$ kubectl exec -it -n $NAMESPACE deploy/tp-backup-controller -- tp-backup -o restore -b <backup name>
   2024-07-25T09:20:20.918Z	INFO	logger/logger.go:52	Logger construction succeeded
   2024-07-25T09:20:20.920Z	INFO	tp-backup/main.go:39	Initialize controller with Loaded Configuration
   2024-07-25T09:20:20.921Z	INFO	storage/azureblob.go:60	Azure Blob storage initialized
   2024-07-25T09:20:20.937Z	INFO	datasource/mariadb.go:49	Wireless datasource Initialized
   2024-07-25T09:20:20.988Z	INFO	datasource/mongodb.go:82	Wireless datasource Initialized
   2024/07/25 09:20:20 connected to 10.240.32.71:2181
   2024/07/25 09:20:21 authenticated: id=144132615539458050, timeout=4000
   2024/07/25 09:20:21 re-submitting `0` credentials after reconnect
   2024-07-25T09:20:22.090Z	INFO	datasource/core.go:80	Core datasource Initialized
   2024-07-25T09:20:22.091Z	INFO	datasource/kubernetes.go:68	Kubernetes datasource Initialized
   WARNING: Destructive Operation !!!
   This Operation will restore a previous state of your TPE instance.
   Are you sure you want to delete current datas? (yes/no)

Overview​

Backing Up​

Manually backing up data​

Scheduling backups​

Restoring data​

Rollback​

Rollback a thingpark-enterprise configuration revision​