Base Station connection issues

Base station is not seen connected by the platform

SYMPTOM: The base station is up and running but the base station is not seen connected by the platform: The base station status remains in "Initialization" or "Connection error" in the TPE GUI.

SOLUTION: See various troubleshooting approaches described below.

Troubleshoot from Cockpit

1. Check that IPsec certificate for Base Station has not expired.

2. If the IPsec certificate has expired, on TPE cockpit, an error message is displayed at the top of the "TPE Configuration" page:

   

3. In this case, you must generate a new certificate and upload it. Refer to the procedure described in the Appendix IPsec certificate Generation for base stations traffic of the ThingPark Enterprise Installation and Configuration Guide.

WARNING

When the certificate is changed, all base stations will be temporarily disconnected to retrieve the new certificate. This new certificate must be regenerated manually for all Base Stations already created. To do it, follow the procedure described Regenerate IPsec certificates for base station traffic.

Troubleshoot from the base Station via Suplog menu

1. Test Connectivity with LRC

Test Connectivity with LRC via "Troubleshooting/Pings LRCs" menu:

If ping fails, you need to test your IPSEC tunnels and connectivity with your PKI IP. For more information, see Test Connectivity with PKI and Test IPSec Status with TPE server.

2. Test Connectivity with PKI

Test Connectivity with PKI via "Troubleshooting/Ping" menu (enter PKI server IP):

If ping fails, check your internal network (firewall, IP flows).

3. Test IPSec Status with TPE server

Test IPSec Status with TPE server via "VPN Configuration/Show IPSec status" menu (not applicable for BS without IPSec):

If you can see "lrc1[1]: ESTABLISHED", then IPSec status is OK.

Otherwise, you need to test if the certificates are correctly installed (see Test if certificates are installed and valid).

4. Test if certificates are installed and valid

Test if certificates are installed and valid via "VPN Configuration/Show IPSec listcerts" menu (not applicable for BS without IPSec):

• Check your UUID in the subject name.
• Check validity date ("expires in XX days").
• If you do not see any output, the certificates have not been installed, you need to check certificates availability from support docker (see Test if BS certificates are available for download).

5. Test if mandatory processes are running

Test if mandatory processes are running via "Troubleshooting/CPU / Memory / Processes" menu:

"lrr.x" application is mandatory (eventually try to reboot the GW if you do not have it).

"Charon" and "checkvpn.sh" processes should be running on GW using IPSEC, if "charon"/ipsec is not running, you need to check certificates availability from support docker (see Test if BS certificates are available for download).

Troubleshoot from Support

Test if BS certificates are available for download

The certificates are stored in Mongo database.

From node1, connect to the mongo docker:

docker exec -it $(docker ps -q -f "name=actility_mongo") bash

Connect to Mongo db:

mongo --username admin --password --authenticationDatabase admin --host localhost --port 27017

Select the "key-installer" database:

rs0:PRIMARY> use key-installer
switched to db key-installer

From "archives" collection, check certificate package availability based on your LRR UUID.

Example below with LRR UID "123456-46584254C0001340".

rs0:PRIMARY> db.archives.find({"lrrUUID": "123456-46584254C0001340"});

If the certificates' package has not been created, try to regenerate the certificate from the TPE GUI.

Troubleshoot from SLRC

Test IPSec status from server side

1. Log IPSec:

   docker logs tpe-slrc

2. Connect inside slrc docker:

   docker exec -it tpe-slrc bash

3. Check IPSec status:

   strongswan statusall

Troubleshoot from LRC

Test GW connection from server side

1. Connect inside lrc docker:

   docker exec -it $(docker ps -q -f "name=actility_lrc1") bash

2. Connect to LRC CLI via telnet (login: support, password: support):

   telnet 0 2009

3. Check all GW currently connected with LRC:

   >> ls

4. Check one GW connection based on its LRR UUID:

   >> ls <LRR UUID>

Troubleshoot from RCA

Test the PKI configuration

1. Connect inside rca docker:

   docker exec -it $(docker ps -q -f "name=actility_rca") bash

2. Check your PKI IP is correct for ipsec configuration:

   cat /usr/local/ejbca/conf/actility-ope/ipsec.conf | grep "right="

Troubleshoot from TPE GUI

Test the connection status reported by the GUI

When VPN and flows between the BS and TPE are all OK, the BS State will change from "Initialization" to "Connection error" for around 10 to 15 min during the first connection.

When the TPE receives the first report from the BS, the status will become "Active".

• If the BS has been deleted and recreated in TPE, the state "Initialization" may persist. In this case, the BS needs to be re-flashed.
• For BS not connected with IPSec, the state "Connection error" may persist. In this case, you need to connect to the BS and update the lrr configuration with the IP (X.X.X.X) of the TPE instance instead of the hostname (hostname.domain).

Base station is not able to establish an connection

SYMPTOM: After an upgrade, base stations are no longer able to retrieve the RFRegion, upload a radio scan or establish a remote connection.

SOLUTION:

Reset the known_hosts file on all the connected gateways (Upgrading to some releases 7.1 may cause new SSH Host Keys to be generated and SSH services to be disrupted):

1. Execute an ssh command to connect to the TPE node:

   ssh support@${IP_OR_HOSTNAME_OF_TPE} -p 2222

2. Then run the following script to reset the known_hosts file:

   /usr/bin/tpe-remove-bs-known-hosts-file