Alarms used for troubleshooting base stations
You can use the following tables to help you troubleshoot base stations. They explain the meaning of each alarm, its probable causes and corrective actions.
System alarms
| Alarm ID / Name | Explanation | Possible Root Causes | Guidelines |
|---|---|---|---|
| 101 / LRR software restarted by watchdog | Base station software gets rebooted by the system protection algorithm via execution of the proper keystrokes. | The alarm gets triggered once the watchdog process stops receiving reporting signals from LRR on its working state within a set period. | Use the remote access to collect logs and contact your support. To learn more, click Performing remote access and rescue access. |
| 102 / Base station connection status | Informs on base station’s connection status to ThingPark LRC-cluster. | Six diverse alarm states occur accordingly to base station’s administrative state (active, validating, suspended) along with base station’s connection state (never connected, disconnected): - The BS is not yet validated by the Administrator (minor alarm by default), - The BS has been suspended by the Administrator (major alarm by default), - The BS is administratively active but has never connected to the core network (warning alarm by default), - The BS is administratively active but is disconnected from the core network (critical alarm by default). | Check the base station power for potential power failure. Check the Ethernet/Cellular connection for potential network failure. Check if the lrr.x process is running, restart it if needed. Check if LRC-cluster has restarted, causing disconnection of the LRR-LRC link. Use the remote access to collect logs and contact your support. To learn more, Performing remote access and rescue access. |
| 106 / Unusually high CPU usage level | CPU usage reaches an abnormal level. | The alarm gets triggered once the CPU’s average and standard deviation cumulated load exceeds 85%: Warning alarm if average standard deviation of the CPU load > 85%, Major alarm if average standard deviation of the CPU load > 95%. | Use the remote access to collect logs and contact your support. To learn more Performing remote access and rescue access. |
| 107 / Unusually high RAM usage level | RAM usage reaches an abnormal level. | The alarm gets triggered once the RAM’s average and standard deviation cumulated load exceeds 80%: Warning alarm if average standard deviation of the CPU load > 80%, Major alarm if average standard deviation of the CPU load > 90%. | Use the remote access to collect logs and contact your support. To learn more Performing remote access and rescue access. |
| 108 / Unusually high file system usage level | File-system usage reaches an abnormal level. | The alarm gets triggered once the file-system average and standard deviation cumulated load exceeds 95%: Warning alarm if average standard deviation of the CPU load > 95%, Major alarm if average standard deviation of the CPU load > 99%. | Use the remote access to collect logs and contact your support. To learn more Performing remote access and rescue access. |
| 109 / Time synchronization lost | base station uses the local time instead of using the NTP time reference. | NTP launched before getting an IP address. The Base station restarted but the NTP was not launched correctly. | Connect on the base station and restart the NTP: settime restart. Use the remote access to collect logs and contact your support. To learn more Performing remote access and rescue access. |
| 110 / Power failure detected | Power supply defect detected; the base station is running on battery. | Base station disconnected from the grid or Power grid failure. | Check the connection of the base station to the grid, Check the power of the grid. |
| 112 / Abnormal log activation | Critical alarm: Log is activated with trace level > 0 without RAM disk usage. Major alarm: Log is activated with trace level > 0 for 7 days (by default). | Log is activated without RAM disk usage, log is activated since several days. | Activate RAM disk usage if not already activated. If an ongoing troubleshooting session does not justify the log activation, deactivate it. If the alarm persists, contact your support if the LRR trace level cannot be set to 0. Note: In some cases when a base station makes many insertions into flash memory, it might damage the normal usage and behavior of the base station. |
| 121 / Backhaul network interface status | At least one backhaul link of this base station is lost. | IP address issue, e.g. due to DHCP. For cellular interface: 3G/LTE network failure, no SIM card, bad network coverage. Bad configuration of the backhaul interface in the LRR. | Use manual IP address allocation in case of DHCP issue. Check the interface configuration in the LRR and correct configuration issues if needed. For alarms related to cellular backhaul check cellular network coverage, SIM card status and subscription with cellular operator. |
RF Cell alarms
| Alarm ID / Name | Explanation | Possible Root Causes | Guidelines |
|---|---|---|---|
| 103 / Unusually low uplink traffic level | No uplink frame for more than X hours; the number X defines the alarm state and can be configured. To learn more, see Uplink activity alarms. | No devices in the LRR covered area. LRR signal interfered by an external jammer. LRR RF chain defect. | Check if there is an active device in the area covered by the base station. Check if an RF equipment has been installed next to the base station (causing potential interference). Scan the radio on the base station to check the noise level. If the alarm persists, contact your Support. |
| 104 / Unusually high level of invalid uplink physical CRC | LoRa® frames received by the LRR (on the last hour) cannot be demodulated properly, or there are many non-LoRa® frames received by the base station with physical preamble like LoRa®. | Number of packets generated by devices in the cell is too high (frame collisions). Noise generated by another LoRa® network's base stations' nearby. Jammer/Non-LoRa® equipment is interfering (most likely if ratio is very high). LRR RF chain defect. | If the uplink duty cycle of the base station is high, Switch off some devices, Check if an RF equipment has been installed next to the base station (causing potential interference), Scan the radio on the base station to check the noise level. If the alarm persists, contact your support. |
| 105 / Downlink frame rate exceeds the RF cell capacity | This alarm is raised when downlink packets are dropped by the base station (received by the LRR but not sent over-the-air) because the RF cell modem was busy. Therefore, the LRR was not able to send these downlink packets (e.g. a downlink packet of a class 'A' device for which the transmission window is outdated). | Too many LoRaWAN® acknowledgments or reconfigurations attempts. | Reduce the acknowledgments requested by your devices. Reduce the downlinks sent by your applications. Have a better repartition of the downlink MAC reconfiguration messages (e.g. RF Region updates) over time to prevent temporary saturation of downlink capacity. Add a new base station to your network. f the alarm persists, contact your support. |
| 111 / Beacon transmission failure | A failure linked to the transmission of the last time-synchronized Beacon from the base station to the Class B device. | Radio is stopped. Listen Before Talk (LBT) procedure did not allow the transmission. GPS synchronization is lost. Beacon is received too late for Beacon slot. Duty cycle constraint is reached. | Check the radio status of the base station. Check the GPS status of the base station. Check the duty cycle on Beacon RF sub-band. Check if the latency between the base station and the Network Server is reasonable. If the alarm persists, contact your support. |
| 113 / Join request replay detected (DevNonce replay) | A repetitive request for a device activation (Join Request) using the same DevNonce has been detected by ThingPark for the same device. | A malicious user: The base station is the victim of a join request replay attack. The device sent a join request containing a DevNonce already used. | If the number of the alarm occurrences is high, Check if the device sent a join request containing a DevNonce already used and fix it, The base station may be the victim of a join request replay attack. Stop the radio and try to identify the attacker. If the alarm persists, contact your support. |
| 114 / Wrong MIC detected in Join request | A wrong MIC has been detected in a Join request, failing to authenticate the activation request for the device in question. | The LRR is the target of a join request replay attack. A device sent a join request containing a wrong MIC possibly due to wrong device provisioning (that is, bad Appkey). | If the number of the alarm occurrences is high, Check the device's information and behavior and fix issues, The base station may be the victim of a join request replay attack. Stop the radio and try to identify the attacker. If the alarm persists, contact your support. |
| 115 / Join request replay detected (wrong MIC correlation) | A wrong MIC correlation has been detected between a Join request and the following Uplink frame, invalidating the last join request received from this device. | If the number of the alarm occurrences is high, the base station may be the victim of a join request replay attack. | Stop the radio and try to identify the attacker. If the alarm persists, contact your support. |
| 116 / Uplink frame replay detected (wrong FCnt) | A wrong FCnt has been detected in an Uplink frame. | The LRR is the target of an uplink frame replay attack. A device reset has not been detected by the Network Server. | If the number of occurrences is high, stop the radio and try to identify the attacker, Reset the security context of the device, if and only if the device has really reset while this reset was not detected by the Network Server. |
| 117 / Wrong MIC detected in Uplink frame | A wrong MIC has been detected in an Uplink frame, failing to authenticate the frame. | The LRR is the target of an uplink frame replay attack. A device sent an uplink frame containing a wrong MIC (for instance, bad NwkSKey provisioned for ABP device). | If the number of the alarm occurrences is high, If the device has sent an uplink packet containing a wrong MIC, fix it (check NwkSKey), The base station may be the victim of a join request replay attack. Stop the radio and try to identify the attacker. If the alarm persists, contact your support. |
| 118 / Uplink frame replay detected (repeated FCnt) | A repeated FCnt has been detected in an Uplink frame. | The LRR is the target of an uplink frame replay attack, A device exceeds its configured number of transmissions. | If the number of the alarm occurrences is high, Stop the radio and try to identify the attacker, Check the behavior of the device and fix it if needed. If the alarms persist, contact your support. |
| 119 / Invalid AppEUI detected in Join request | An invalid AppEUI has been detected in a Join request. | The AppEUI value has changed too many times for this device. | If the device has used more than 32 different AppEUIs, fix it. If the alarms persist, contact your support. |
| 120 / Join request replay detected (invalid DevNonce) | A DevNonce attack has been detected in a Join request when the device is supposed to use counter-based DenNonce. | If the number of the alarm occurrences is high, the base station may be the victim of a join request replay attack. | If a join request replay attack: stop the radio and try to identify the attacker. If the alarms persists, contact your support. |