22nd December, 2023
Issue:
We have had reports of some Android 10 devices failing to boot and entering Rescue Party mode.
Cause:
This is caused by an old bug (referred to here as the ‘uninstallation bug’) that was fixed in November 2019 but is still present in devices that haven’t been updated to newer versions of Android.
The bug manifests when uninstalling updates to a system app, and the uninstall operation fails or is interrupted. It is not clear what causes the failure, but given the low incidence of the bug it could be due to a race condition.
Trigger:
Google has been running an operation to recover devices that were affected by another bug (the ‘Play Store bug’), and we believe this operation was the trigger for the uninstallation bug affecting devices now.
The recovery operation uninstalled the update to the Play Store, restoring it to the system version and allowing it to self-update to the correct version.
During our internal testing and through the initial rollout phases of the recovery operation we saw no incidences of the uninstallation bug occuring. The operation was enabled for remaining devices on December 18th.
Mitigation:
We received the first reports of devices being affected by the uninstallation bug on the evening of December 19th and the Play Store recovery operation was halted the next morning, removing the trigger. There are no plans to resume the operation.
Fix:
We have advised device manufacturers to apply the original fix to their builds. Flashing a new system image that includes the patch has been confirmed to recover devices that were stuck in the Rescue Party. As the fix was included in Android 11, flashing the devices to an Android 11 system image should also recover affected devices.
Factory resetting affected devices will restore the device to working order.
We also recommend that organizations upgrade to newer versions of Android in general.
