AlarmNet360 Posts

The cyber incident on Friday July 19, 2024 was caused by a code error in an update pushed to Windows machines by CrowdStrike. This was not a malicious attack. CrowdStrike Falcon, the specific product impacted, is a cloud-based product with a small local footprint, designed to detect breaches.

CrowdStrike Falcon is a cloud-based protection product. CrowdStrike describes the product this way: "Falcon is the CrowdStrike platform purpose-built to stop breaches via a unified set of cloud-delivered technologies that prevent all types of attacks — including malware and much more."

A small file, termed a sensor, is installed on the computer. This sensor monitors for viruses, malware, zero-day (emerging), and other types of threats. The sensor communicates with CrowdStrike through the cloud, and if a breach is detected, CrowdStrike can then respond. By keeping the bulk of the service in the cloud, the protected computer isn't bogged down with a resource-heavy software package. The sensor file is only about 5 MB.

CrowdStrike recommends that their Falcon customers use an N-2 update cadence, or at least N-1. This means the sensor file software runs either one update (N-1) or two updates (N-2) behind the current version. Ideally, this allows any issue with an update to be found and resolved before it ever reaches a client computer.

The update that caused the Windows Blue Screen of Death (BSOD) and boot-looping issue last Friday wasn't controlled by the N-1 or N-2 policy that is set up on most systems. The update was to the signature files, which help the Falcon Sensor determine what is a threat, and they need to be updated as quickly as possible. For this reason, they aren't covered by the usual update cadence.

The U.S., Canada, the UK, Europe, and Asia experienced disruptions to various services during the outage. While Mac and Linux computers remained unaffected, over 4,000 flights worldwide were canceled. The financial and healthcare sectors were severely impacted, with many elective medical procedures postponed. Numerous payment systems were also unavailable in the early hours of the incident.

There's much more information about this available online if you want to find it. CrowdStrike has been very transparent in dealing with this issue. But what we're discussing is why this issue manifested the communication troubles some of you saw last week. For that, we'll talk about how alarm communicators are supervised.

Since this affects both AlarmNet and Alarm.com, I'm going to speak in generalities. I reached out to both Alarm.com and AlarmNet (Resideo) for specifics, but since I haven't heard back, I'm going to provide the general information I know about this process.

When an alarm system has a signal to report, it does so using whatever channel or channels are available to it. This could be through a POTS phone line, a WIFI or Ethernet connection, an LTE or LTEM Cellular connection, or some combination of these paths. The important thing to know is that when an alarm panel sends a signal, it looks for an acknowledgment that the signal was received successfully. If it doesn't receive that acknowledgment, it will send the signal again (and again) until eventually it either reaches the destination, the retransmission limit, or the time limit, depending on the path used.

At the other end of this communication is the Alarm.com or AlarmNet server. This system receives those signals and processes them. This may include forwarding the information to a central station, to an online platform for logging and distribution to the end-user, or both.

Since the system may never have an alarm, there are measures in place to send periodic test messages from the alarm system communicator to the server. This ensures that all aspects of any communication pathways are open and working. This usually involves setting a communication test interval. For cellular communication in particular, it's desirable to minimize unnecessary signals, so this is customarily a "smart" test.

For example, you may set a system for a daily or 24-hour test. This is a setting at both the alarm panel and the signal processing server. That means every 24 hours, the system will send a test message to the server to verify communication. If no test message is received, the server generates a central station message that the system failed to properly test.

With a "smart" test, any signal sent by the system resets the test timer. So, the only time the server will receive an actual test message is if the system hasn't sent any other type of signal for 24 hours. In either case, based on this example, if the signal processing server goes 24 hours without receiving either a regular signal or a test signal, a trouble condition is generated.

From here, we can only assume that the signal processing server or servers were impacted by the CrowdStrike Falcon update. We can safely assume this because Verizon and AT&T LTE cellular communications were not affected by this issue. The way I see it, this incident was a blessing in disguise.

Though this probably seems like a catastrophic event, it's actually an opportunity. Because this was not a malicious attack, the least possible harm has come from it. Those with robust disaster recovery plans got a real-world chance to put them to use. Those without robust disaster recovery plans now know what's at stake and can plan accordingly. Catalysts for change and improvement are rarely painless, and this is no exception.

It was only supposed to be a simple software update! These words are I'm guessing, being echoed by the cybersecurity giant, CrowdStrike. Over night CrowdStrike pushed out a misconfigured/corrupted update to its customers. This has caused major issues to banks, airports, TV stations, health care organizations, hotels, and you guessed it, the alarm industry.

Early this morning, Alarm Grid became aware of multiple issues concerning alarm systems. Resideo reported issues July 19 at 3:57am EST. They reported All Pro Series and LTEM-P devices failing on all services. Also, TC2 Application notifications and event processing were delayed. They have reported that most systems are already back online.

Alarm.com reported issues July 19 at 1am EST. The reported issues were affecting access to the Partner Portal, Mobile Tech, the customer website, and the customer app, and may cause delayed execution of commands and signals sent to and from customer systems.

CrowdStrike has reported that the fix has been implemented, but that it could take time to get things back up and running. Both AlarmNet and Alarm.com services are reporting improvements.

Please continue to check back here for updates.

UPDATE: As of 10am EST, Alarm.com has reported that all of their services are fully restored.

UPDATE: 3:30p EST. Resideo has reported that most of the systems are back online. And that alarm delivery services are not impacted. Although we saw their service return much earlier than this update.

Thanks for your patience. If you continue to have issues with Alarm.com or Resideo's services, you can email us here.

Recently, while writing content about the installation of various add-on modules for the PROA7PLUS, I discovered the "Shutdown system" option. Being familiar with Resideo's definition of "Shutdown" I sent an email to support to find out exactly what this button does. The answer surprised me.

I was looking for a better way of powering down the panel for the purpose of adding hardware. Having to disconnect the battery and possibly unplug the DC power adapter is inconvenient and unnecessarily difficult. I was hoping for a menu option that would allow me to power down without physically disconnecting anything.

When you enter the menu options of the PROA7PLUS, PROA7, PROA7PLUSC, or PROA7C you see an option for Tools. If you select Tools, you'll be prompted to enter a code. Once you enter a valid code, and this includes both the Installer Code and the Master Code (4112 and 1234 respectively, by default) you'll then see a list of additional options.

At the bottom of this list is "Shutdown system". I was hoping this was an option to power down the system. Just above it in the menu list is "Reboot System" but rebooting doesn't help when you're attempting to add or remove hardware. Again, knowing Resideo as I do, I was afraid to simply try out the shutdown option, so I sent an email to their technical support to inquire about what this menu option does. I found the answer a little disturbing.

According to support, when you perform a system shutdown the system cannot be armed. Only 24-Hour emergency zones can be activated, and those alarms are local only because the system's communicator is also shut down. The burglary portion of the system is useless in this state. Access to the "Tools" menu is also disabled.

You might wonder, as I did, if the communicator is shut down, and the "Tools" menu is disabled how the heck do you recover from this state? According to technical support, it is still possible to send a command to the system from AlarmNet360. This is called a "Resume System" command. This option can only be performed by your alarm dealer.

My takeaways from this are: If your system is not registered with an alarm dealer DO NOT SELECT SHUTDOWN SYSTEM from the "Tools" menu. Doing so will likely cause your system to be permanently inoperable. It's unclear if an alarm dealer could register a system after the shutdown command has been implemented, and then send the Resume System command. I wouldn't recommend testing it.

For anyone who may have accidentally hit this menu option and who does have a registered system, contact your alarm dealer immediately. Let them know what happened and that you need to initiate a "Resume System" command through AlarmNet360. They should be able to help you get your system back up and running.

If you're an Alarm Grid customer who finds yourself in this position you can email us at support@alarmgrid.com, or call 1-888-818-7728. Our support team is here Monday - Friday from 9:00 am - 8:00 pm, with the exception of holidays. We will hopefully be able to send the command to Resume System using AlarmNet360.

We first told you about the First Alert® VX5 Indoor Camera in our ISC West blog earlier this year. The VX5 debuted at the trade show. However, it's still not available for purchase. That's pretty routine, announcing a product that nobody can get yet. But, here's what we know about it so far.

For one thing, it looks like that ⬆. It is attractive and unobtrusive. This is an indoor camera that can be mounted to the ceiling, wall, or can sit on any flat stable surface. It requires a wired connection to power, but is otherwise wireless. Using Total Connect 2.0, live video and clips can be viewed from any location where the user has access to either a cellular or WIFI connection to the internet.

Advanced Event Detection

The VX5 Indoor Camera offers advanced event detection using state-of-the-art AI. It can tell the difference between a person, animal, or vehicle, and can alert you to the same. This AI capability will no doubt pay dividends down the road, and I can foresee new features being added that also capitalize on it.

Offers Video and Audio Capabilities

The VX5 is capable of 2MP resolution, which gives sharp, detailed images in both day and night vision situations. The ability to both hear and communicate with anyone in the camera's vicinity allows the user to engage with those being viewed. This can be used as a deterrent if you catch a person (or animal) lurking in an area where they don't belong.

Automated Privacy Mode

Privacy is on everyone's mind these days. The VX5 offers both a manual privacy setting, as well as automated privacy settings. Privacy Mode can be selected from the Total Connect 2.0 app or by pushing a button on the camera itself. Privacy Mode prevents the camera from recording any video or audio. When integrated with one of the PROSeries security panels, the privacy shield will come on automatically when the system is disarmed and turn off when the system is armed.

Remote Access Using Total Connect 2.0

The new First Alert® VX5 Indoor Camera leverages the user's Total Connect 2.0 subscription, giving remote access and notification capability to any of the VX cameras associated with the account. For Alarm Grid customers, a Platinum Plan (Self or Full) will support up to eight (8) VX cameras with 30-days of recording capability. If you need more than eight (8) cameras, you can add a Video-Monitoring Plan, which doubles this support.

In Summary:

The First Alert® VX5 Indoor Camera is a much-needed addition to Resideo's Total Connect 2.0 camera lineup. It offers a fresh look and modern features that consumers want in a camera. They're taking a "one-size-fits-all" approach so far, so let's hope this camera performs in the way we expect.

This camera is still considered to be in the Beta stage of development. That means:

• It could still be a while before it's released.
• Anything we've been told about it so far could be changed before it's released.

One thing that's not likely to change is how it looks, so the picture above should be accurate. As for everything else, we'll just have to wait patiently for it to be released then hopefully, we'll see.

Update 04/08/24 5:08 pm: According to a contact at Resideo, the available stock of IPCAM-WOC2 cameras without this issue is being held back to use as replacements for any affected units that have already been sold. Stock is limited, and once it is depleted, the camera will likely be discontinued permanently.

As an alternative, Resideo recommends the First Alert CAMWE-WO, also known as the VX3 HD Camera. This is an outdoor, 1080p camera with Wide Dynamic Range (WDR). With its intelligent event detection, the CAMWE-WO can distinguish between vehicles, animals, and people.

Original Post: If you have purchased a new Resideo IPCAM-WOC2 recently, but have not yet tried to register it, there is a good chance it will not register. A manufacturing issue has been identified, and a fix is in the works. However, you will need to warranty replace the existing camera if affected

Resideo released this notice today regarding the issue. The notice is a bit confusing. It states that because of this issue they have decided to stop selling the IPCAM-WOC2, but it also states that a resolution has been identified and that new product is available in their warehouse and that affected units can be returned for exchange or refund.

What we know currently:

• Installed, registered, and working units are not affected. There is no reason to believe they will have future issues related to this announcement.
• Units that have been purchased but not registered may have the issue. If you attempt to register an IPCAM-WOC2 and it fails, return it to the vendor you purchased it from for an exchange or refund.
• Currently, you can't purchase a new IPCAM-WOC2. Presumably, current stock of working units is being used for warranty replacement.

We've reached out to Resideo to try and get clarification on a few of our questions. When we hear back, we'll update this blog post. In the meantime, if you have purchased an IPCAM-WOC2 and you haven't tried to register it yet, it is recommended that you do so now. If you encounter an issue, contact the distributor or dealer you purchased the unit from and proceed from there.

If you are an Alarm Grid monitored customer, or you purchased an IPCAM-WOC2 from Alarm Grid recently, you can contact us regarding a warranty replacement. Send an email to support@alarmgrid.com. If the email you use to contact support is different from the email you used when ordering your product, please provide us with the ordering email. That is the easiest way for us to look up and confirm your recent order.

UPDATE 3:00 PM EST 02/22/24:

The nationwide AT&T outage has been resolved and all affected users, including alarm communicators, have been restored to normal operation. AT&T is still investigating the cause of the outage, and no concrete information is yet available. If a forensic analysis is made public, we will comment on it in a separate post.

Both AlarmNet and Alarm.com show that service for customers with AT&T communicators has been fully restored. If you have an AT&T communicator that still seems to be offline, attempt to power cycle the device.

Always put a monitored system on test with the monitoring station before performing any kind of maintenance.

For most stand-alone communicators, you can do this by unplugging the battery and the primary power supply, waiting about 30 seconds, and then powering back on by plugging the battery, and then the power supply back in. If your communicator gets its power from a VISTA alarm panel, like the LTEM-XA, then power cycle the entire system. Power down by unplugging the panel's battery and transformer. Power up by plugging in the transformer first, and then the battery.

For older All-in-One systems, like the LynxTouch panels, you'll need to power cycle the entire system. Do this by opening the panel and unplugging the battery, then unplugging the transformer. When powering back up, plug the battery in first, then the transformer.

The PROA7 and PROA7PLUS panels have a reboot option through the Tools Menu. From the Home Screen, tap Menu (≡) > scroll down, tap Tools > Enter Installer Code (4112 by default) > scroll down and tap Reboot.

If you have a Qolsys touchscreen system, be sure to go through the panel's menu to reboot, rather than removing the battery and DC power. Not following the proper power-down or reboot procedure can damage the panel. Tap the gray bar at the top of the screen, then tap Settings > Advanced Settings > Enter Installer or Dealer Code (1111 or 2222 respectively, by default) > Panel Reboot (↻).

ORIGINAL POST:

A nationwide outage, beginning this morning at approximately 3:30 AM, is affecting both mobile phones and alarm communicators. The outage seems to be limited to AT&T, though there were early reports of issues with T-Mobile also. According to T-Mobile, their service is working normally now.

AlarmNet has posted this notice on the AlarmNet360 page:

Alarm.com has posted this notice:

It is important to note that this is a carrier outage, not an AlarmNet, Total Connect 2.0, or Alarm.com outage, meaning this is not something Resideo or Alarm.com can resolve. As new information becomes available, we will update it here. As an AT&T Subscriber in the Louisville, KY area, I can report that I have no cellular service at this time. AT&T recommends that their wireless users take advantage of WIFI calling wherever possible, until this issue is resolved.

This outage does potentially affect your ability to remotely control your system, to receive notifications from your system, and your system's ability to report alarm and other signals to a central station, but only if you have an AT&T communicator tied to the system or have an AT&T Mobile Device and no access to WIFI. To our knowledge, Verizon communicators are unaffected.

If your system reports a trouble condition related to the cellular outage, you can silence any audible alert by acknowledging the trouble condition. For users with touchscreen panels, this will usually involve touching the notification message and then acknowledging it on the next screen.

For VISTA users, a disarm command should be entered. This is accomplished by entering a valid 4-digit user code, and then pressing the 1 or OFF key. Once the outage has been restored, you will need to perform a disarm command again to clear the trouble condition from the display.

Most wired DSC panels indicate a trouble condition by displaying a triangle with an exclamation point inside. Pressing [*] [2] and then viewing the number or numbers displayed will explain what the condition is. A four usually (4) indicates a Comm Failure. You can see information about how to view DSC trouble conditions here. DSC Impassa users can see the Install Guide here. Information about trouble codes is on page 72. You can see a DSC PowerSeries troubleshooting guide here.

02/14/24 1:09 PM UPDATE: Per Resideo, these two issues are unrelated. However, the second issue, involving AT&T communicators appears to have been resolved.

02/13/24 4:15 PM UPDATE: Resideo has an update on the AlarmNet360 site currently:

"A small percentage of ProSeries and LTEMP LTE AT&T communicators may experience signaling issues that could result in a communication failure message being generated to the monitoring Central Station.

This issue is being worked with AT&T for immediate resolution.

We appreciate your patience while we work to quickly resolve this issue."

It is not clear whether this is a separate issue, or is related to the Pro Series issue originally reported in this post. When we get that information, we will either update here or in a new post with details of the new issue. Keep watching this space for updates.

02/13/24 Original Post:

Recently, Resideo identified an issue with select PROA7 and PROA7PLUS panels: screens become dark/unresponsive, losing connection to AlarmNet. A hard reboot is required for a reset. Resideo plans a firmware upgrade to address this, which will be applied automatically to all Pro Series panels.

This alert applies to all Resideo and Honeywell Home Pro Series panels. This includes the PROA7, PROA7C, PROA7PLUS, and PROA7PLUSC. While they say the number of affected panels is very small, we don't yet have an actual number or a date code range to confirm those details.

In response to this issue, Resideo has a firmware update, 03.1872.18.0 which they will begin pushing to all registered Pro Series panels this week. The update will also be available to dealers to push manually beginning on 02/13/24. This is a critical update and will be pushed over WIFI, if available, or Cell if WIFI is unavailable. It is important to note that only registered panels can receive this firmware update. Registered means that there is an account associated with the panel.

An affected panel will be dark, with no status LEDs illuminated. Along with these visible symptoms, the panel will stop communicating with the AlarmNet servers. This will eventually generate a report to the monitoring station of E359, which indicates the panel failed a regularly scheduled check-in.

If you have a Pro Series panel and believe your panel is affected, we recommend you power cycle it as soon as possible. Remove the set screw in the bottom of the panel and pull the front and back of the panel apart. Doing this will automatically remove the primary DC power.

Next, find the panel's battery and unplug it from the main board. Leave it disconnected for about 30 seconds, then plug it back in. Now carefully put the panel back together. Doing so will reconnect the primary power and the panel should begin to boot up. Replace the set screw, if installed and allow the panel to boot. This should return the panel to normal operation and allow it to receive the firmware update.

Panels that receive the firmware update will reboot as a part of the update process. This will generate a signal to the monitoring station of E-339-Exp. Module Power On / Reset, or possibly E-305-System reset, or possibly both.

If we receive further details on this issue, we'll post them here, so stay tuned. Updates will be posted near the top of this page and will be highlighted with a bold date and wording to indicate new information. We have yet to receive an official MWT Technical Bulletin on this issue, but I believe one is likely coming soon. Hopefully, it will contain the details that are, so far, notably missing.

According to Resideo Technical Support, their integration with Accuweather is partially down. This may cause products that receive weather information some issues. Any touchscreen product that can display weather may show incorrect information. Any automation configured for sunrise/sunset may not operate.

We received this information based on a support request we sent to Resideo. In that specific case, automation schedules based on sunrise and sunset that had been programmed through Total Connect 2.0 were not working correctly. So far, there's been no official report of this outage in AlarmNet360. We are hopeful that additional details will be forthcoming.

Do products or TC2 show incorrect weather in addition to the sunrise and sunset timers being off? We don't know for sure. Is this a nationwide issue, or a regional one? We don't know that for sure either. If you have more details or would like to share your experience, leave a comment below. It could be helpful.

Resideo has not had great success with its products and displaying the weather. In the past, we've reported on issues with the older style Tuxedo keypads not updating the weather properly. Hopefully, this is something that will be resolved in due time. In the meantime, if you have automation schedules for your devices that are set to run at sunrise and/or sunset, be aware that they may not function as intended.

10/6/2022: As of now, Resideo has not posted an update showing that this issue has been resolved. They do report that some customers have had success in deleting the SkyBell from Total Connect 2.0, and then re-enrolling it.

Resideo announced today that users of their Total Connect 2.0 service may currently be experiencing impairment in service. This is as of 1:15 PM Eastern Time today. The services affected are TC2 SkyBell Live Streaming, TC2 SkyBell Clip Replay, and TC2 Video Event Capture.

These services are basically the full range of things you might use a video doorbell for. Bear in mind that TC2 and SkyBell use a server-to-server integration. That means that all the heavy lifting with regard to the SkyBell integration is happening on the SkyBell servers. The TC2 servers are just sending requests back and forth to those SkyBell servers. We don't know if this issue is completely on the SkyBell side of things, or if there is a breakdown in communication between the TC2 servers and the SkyBell servers. If any more information comes to light, we'll share it here.

In the meantime, what we do know is that all other aspects of Total Connect 2.0 service are working as they should. Also, and most importantly, the delivery of alarm signals is not in any way impaired by this temporary situation. We know that the issues began at 1:15 PM ET, and as of now, the outage is still ongoing. If we receive information on an end to this situation, we'll post it here, so stay tuned.

Verizon plans to shut down its CDMA (3G) network by December 31, 2022. If you have a Verizon CDMA communicator on your system, you need to upgrade by then, or your system may be unable to communicate in the event of an emergency. Alarm Grid is making the upgrade as painless as possible.

We've posted about the AT&T 3G and Verizon CDMA sunsets a lot! Verizon CDMA users have been fortunate in that they've had nearly a year longer to update than AT&T 3G users. However, the sunset is drawing near, so those who still have a system with a CDMA communicator should be giving serious consideration to upgrading. If you're not sure how to upgrade, this post provides great information.

If you're concerned about the cost of upgrading, we offer financing through Affirm. From this link, choose "Shop Now" and follow the prompts to apply at check out. If you have any questions, contact us at support@alarmgrid.com if we can be of assistance, we're more than happy to help.

You might wonder why we continue to talk about upgrading your system. It's because we know that cellular alarm communication is the most reliable method of communication available today. Internet, be it Ethernet or WIFI is great. It's fast, and when it's available it makes using services like Alarm.com or Total Connect 2.0 much more fluid. It's also very fast for reporting alarm signals when central station monitoring is in use.

But we all know that an internet connection can be unreliable. Sometimes WIFI just stops working and requires a reboot or other end-user intervention. For most folks, when the power goes out, internet equipment also goes down. This means that even a wired Ethernet connection to the internet can be vulnerable to downtime.

With cellular, those types of issues are much less likely to occur. Cellular providers have all sorts of redundancy built into their networks. So, a loss of power is countered by backup power sources for cellular towers. The spacing of the towers themselves means that in most cases, a cellular device will have a primary tower and at least one backup tower to keep it connected and communicating. Cellular alarm communicators either have their own backup battery or use the battery of the alarm panel they are connected to. So the loss of primary power won't take the communicator offline.

Finally, we'll wrap this up by reminding Alarm Grid customers that waiting until the last minute to upgrade your communicator is a bad idea. We have a finite number of activators, and the holiday season is coming up soon, which will further limit the time we have to get new communicators activated. Upgrading from an older communicator to a new one usually doesn't take quite as long as a new activation, but it does require manual input from one of our activators, so please be mindful of this and schedule your upgrade using our communicator replacement portal as soon as possible.