The battery cover or battery compartment of a Honeywell thermostat is not labeled, leaving many consumers unsure of replacing it. While this information can easily be found in the owner's manual, it is often discarded after purchase. If you have a Honeywell digital thermostat, it will probably have a monitor that notifies you when it has a low battery.