In the latest Cisco Talos Incident Response Quarterly Trends report, instances related to multi-factor authentication (MFA) were involved in nearly half of all security incidents that our team responded to in the first quarter of 2024. 

In 25% of engagements, the underlying cause was users accepting fraudulent MFA push notifications that originated from an attacker. In 21% of engagements, the underlying cause for the incident was a lack of proper implementation of MFA. 

I was curious to see what some of the reasons might be as to why these two issues were the top security weaknesses outlined in the report. To do so, I’ll explore (with the help of Cisco Duo’s AI and Security Research team and their push-based attack dataset) the parameters that attackers are using to send their fraudulent MFA attempts, including: 

• The percentage of MFA push spray attacks accepted by the user. 
• How many push requests a victim user was sent. 
• Peak times of occurrence. 
• Time between successive push attempts. 

I’ll also explore the current methods that attackers are using to bypass MFA or social engineer it to gain access.  

It’s worth noting that there has been a lot of progress made by defenders over the past few years regarding implementing MFA within their organizations. MFA has significantly contributed to reducing the effectiveness of classic credential stuffing and password spraying attacks by adding an extra layer of authentication. This is a large reason as to why attackers are targeting MFA so heavily – it’s a significant barrier they need to get around to achieve their goals.  

But as with any form of defense, MFA isn’t a silver bullet. The issues we’re seeing now are mostly down to attacker creativity to try and bypass MFA, and overall poor implementation of the solution (for example, not installing it on public-facing applications or EOL software). There are also some legitimate cases where MFA cannot be implemented by an organization, in which case, a robust access policy must be put in place. 

The data behind push spray attacks 

The most common type of MFA bypass attempts we see are MFA push attacks, where the attacker has gained access to a user’s password and repeatedly sends push notifications to their MFA-enabled device, hoping they will accept. 

We asked Cisco Duo’s AI and Security Research team to provide some metrics for push-based attacks from their attack dataset, which contains 15,000 catalogued push-based attacks from June 2023 - May 2024.  

In the first metric (the overall response to fraudulent pushes) we learn that most push-based attacks aren’t successful i.e., they are ignored or reported. Five percent of sent push attacks were accepted by users. 

However, of that 5%, it didn’t take many attempts to persuade the user to accept the push. Most users who accepted fraudulent pushes were sent between one and five requests, while a very small number were “bombarded” with 20 - 50 requests. 

The team also looked at the times of day when fraudulent push attempts were sent. The majority were sent between 10:00 UTC and 16:00, which is slightly ahead of U.S working hours. This indicates that attackers are sending push notifications as people are logging on in the morning, or during actual work hours – presumably hoping that the notifications are in context of their usual working day, and therefore less likely to be flagged. 

There is a large peak between 8 and 9 a.m. (presumably when most people are authenticating for the day). The small peak in the early evening is less clear cut, but one potential reason is that people may be on their phones catching up on news or social media, and may be more susceptible to an accidental push acceptance. 

Most authentications within a single push attack (sent from the same classified IP) occurred within 60 seconds of each other. As authentications timeout after 60 seconds, the most common “failure” reason was “No response.” 

Rather than a “spray-and-pray” approach, this data appears to indicate that attackers are being more targeted in their approach by sending a small number of push notifications to users within a certain period. If they don’t respond, they move onto the next user to try and target as many users as possible within the peak time of 8 – 9 a.m. 

Different examples of MFA bypass attempts

As well as push-based spray attacks, recently we have seen several instances where attackers have got a bit creative in their MFA bypass attempts.  

In speaking to several members of our Cisco Talos Incident Response team, here are some of the MFA bypass methods that they have seen used in security incidents, beyond the “traditional” MFA push-spray attacks: 

1. Stolen authentication tokens from employees. Attackers then replay session tokens with the MFA check completed (giving the attackers a trusted user identity to move laterally across the network). 
2. Social engineering the IT department to add new MFA enabled devices using the attacker’s device. 
3. Compromising a company contractor, and then changing their phone number so they can access MFA on their own device. 
4. Compromising a single endpoint, escalating their privileges to admin level, and then logging into the MFA software to deactivate it. 
5. Compromising an employee (otherwise known as an insider attack) to click “allow” on an MFA push that originated from an attacker. 

The attacks outlined above don’t solely rely on MFA weaknesses – social engineering, moving laterally across the network, and creating admin access involves several steps where red flags can be spotted or ultimately prevented. Therefore, taking a holistic view of how an attacker might use MFA or social engineer their access to it is important. 

New MFA attack developments 

As the commercialization of cybercrime continues to increase with more attacks becoming available “as a service,” it’s worth paying attention to phishing-as-a-service kits that offer an element of MFA bypass as part of the tool. 

One such platform is the Tycoon 2FA phishing-as-a-service which relies on the attacker-in-the-middle (AiTM) technique. This isn’t anything new – the technique involves an attacker server (also known as reverse proxy server) hosting a phishing web page, intercepting victims’ inputs, and relaying them to the legitimate service.  

The tool has now incorporated the prompt of an MFA request. If the user accepts this, the server in the middle captures the session cookies. Stolen cookies then allow attackers to replay a session and therefore bypass the MFA, even if credentials have been changed in between. 

Cat and mouse

These push spray attacks and MFA bypass attempts are simply an evolution of cybersecurity defense. It’s the cat-and-mouse game that persists whenever defenders introduce new technology. 

When defenders introduced passwords, attackers introduced password-cracking methodology through rainbow tables, tools like Hashcat and GPU cards. Defenders countered this by introducing account lockout features. 

Attackers then introduced password spray attacks to obtain credentials through dedicated tools such as MSOLSpray. After that, defenders brought out MFA to add an additional credential check. 

Next, attackers developed dedicated tools like MFASweep to find gaps in the MFA coverage of organizations, looking for IP addresses and ranges, or specific OS platforms that are granted an exception. MFA bypass also contributed to a comeback of social engineering techniques. 

With the MFA bypass attempts that are happening in the field, defenders are now exploring various countermeasures. These include WebAuthn and inputting a four-digit number code into MFA tools such as Cisco Duo (requiring the user to input specific text is a stronger MFA method than say SMS). And considering a Zero Trust environment to include contextual factors, such as where and when the device is accessing the system. 

Recommendations

From an organizational/ defender point of view, here are some of Talos’ recommendations for implementing MFA: 

• Consider implementing number-matching in MFA applications such as Cisco Duo to provide an additional layer of security to prevent users from accepting malicious MFA push notifications.  
• Implement MFA on all critical services including all remote access and identity access management (IAM) services. MFA will be the most effective method for the prevention of remote-based compromises. It also prevents lateral movement by requiring all administrative users to provide a second form of authentication.  
• Organizations can set up an alert for single-factor authentication to quickly identify potential gaps and changes in the MFA policy (if for example, MFA has been downgraded to a single factor authentication).  
• Conduct employee education within the IT department to help prevent social engineering campaigns where attackers request additional MFA enabled devices or accounts. 
• Conduct overall employee education about MFA bypass attacks and how they may be targeted. Provide clear reporting lines for alerting the organization to potential MFA attacks. 
• In cases where MFA cannot be implemented, for example on some legacy systems that cannot be updated or replaced, work with your MFA vendor to define access policies for those systems and ensure they are separated from the rest of the network. 
• Another potential authentication method is a Security key – a hardware device that requires a PIN. 

Read the latest Cisco Talos Incident Response Quarterly Trends report to learn more about the current threat trends and tactics. 

Read the Cisco Duo Trusted Access Report to examine trends (existing and emerging) in both access management and identity. 

We’ve been discussing networking devices quite a lot recently and how Advanced Persistent Threat actors (APTs) are using highly sophisticated tactics to target aging infrastructure for espionage purposes. Some of these attacks are also likely prepositioning the APTs for future disruptive or destructive attacks. 

Talos has also observed several ransomware groups gaining initial access to networking devices to extort their victims. We wrote about these attacks in our 2023 Year in Review report. 

The mechanisms and methodology behind these two groups are drastically different, but no less concerning. This is partly because networking devices offer a great deal of access to an attacker. If you can compromise a router, you are highly likely to have a point of ingress into that network.  

These attacks are largely being carried out on aging network infrastructure; devices that have long since gone end-of-life, and/or have critical unpatched vulnerabilities sitting on them. Many of these older devices weren’t designed with security in mind. Traditionally, network infrastructure has sat outside of security’s ecosystem, and this makes monitoring network access attempts increasingly difficult. 

Adversaries, particularly APTs, are capitalizing on this scenario to conduct hidden post-compromise activities once they have gained initial access to the network. The goal here is to give themselves a greater foothold, conceal their activities, and hunt for data and intelligence that can assist them with their espionage and/or disruptive goals. 

Think of it like a burglar breaking into a house via the water pipes. They’re not using “traditional” methods such as breaking down doors or windows (the noisy smash-and-grab approach) — they’re using an unusual route, because no one ever thinks their house will be broken into via the water pipes. Their goal is to remain stealthy on the inside while they take their time to find the most valuable artefacts (credit to my colleague Martin Lee for that analogy). 

In this blog, we explore how we got here, and the different approaches of APTs vs ransomware actors. We also discuss three of the most common post-compromise tactics that Talos has observed in our threat telemetry and Cisco Talos Incident Response (Talos IR) engagements. These include modifying the device’s firmware, uploading customized/weaponized firmware, and bypassing security measures. 

How we got here

There is a rich history of threat actors targeting network infrastructure — the most notorious example being VPNFilter in 2018. The attack was staged, but potential disaster was averted when the attacker’s command and control (C2) infrastructure was seized by the FBI, preventing the attacker from issuing the final command to take over the devices. 

At the time, we spoke about how VPNFilter was the “wakeup call that alerted the cybersecurity community to a new kind of state-sponsored threat — a vast network of compromised devices across the globe that could stow away secrets, hide the origins of attacks and shut down networks.” 

The techniques used in VPNFilter gives us plenty of clues as to possible current threat actor motivations. In the attack, the modular design of the malware allowed for many things to take place post compromise – one module even allowed the malware to create a giant Tor network of the 500,000 compromised devices.  

A recent attack which may have been inspired by this was the KV Botnet (Lumen released a blog about this in December 2023). The botnet was used to compromise devices including small and home office (SOHO) routers and firewalls and then chain them together, “to form a covert data transfer network supporting various Chinese state-sponsored actors including Volt Typhoon.”  

The Beers with Talos team recently spoke about the KV Botnet and Volt Typhoon, a group widely reported by the U.S. Cybersecurity and Infrastructure Security Agency (CISA), Microsoft and other organizations to be a PRC-based state actor. They have been known to conduct long-term espionage activities and strategic operations that are potentially positioning them for future destructive/disruptive attacks. Listen to the episode below:

In 2019, we saw another type of modular malware that was designed to target network infrastructure: “Cyclops Blink.” This was dubbed the “Son of VPNFilter” because of the similarities to that campaign.

The Cyclops Blink malware was designed to run on Linux systems, specifically for 32-bit PowerPC architecture. It could be used in a variety of ways, including reconnaissance and espionage activity. It leveraged modules to facilitate various activities such as establishment of C2, file upload/download and data extraction capabilities.  

In 2022, Talos wrote about how we had detected compromised MikroTik routers inside of Ukraine being leveraged to conduct brute force attacks on devices protected by multi-factor authentication. This continued the pattern we have seen since our investigation into VPNFilter involving actors using MikroTik routers. 

For more insights into the status of attacks on network infrastructure, here is Talos’ Matt Olney and Nick Biasini talking about what Talos has observed over the past 18 months: 

Post compromise tactics and techniques

Compromising the network for persistent access and intelligence capture is a multi-step process and requires a lot of work and expertise in targeted technologies which is why we typically only see the most sophisticated threat actors carry out these attacks.  

Below are some techniques that Talos has observed post compromise on out-of-date networking equipment, in order to maintain persistent access. We initially discussed these in our threat advisory in April, as well as our 2023 Year in Review, but due to the sophisticated nature of these attacks and the continued exploitation, we wanted to dive deeper into some of these tactics: 

1) Modifying the firmware

Talos has observed APTs modifying network device firmware on older devices to add certain pieces of functionality, which will allow them to gain a greater foothold on the network. This could be adding implants or modifying the way the device captures information.  

An example of this is the recent exploitation of Cisco IOS XE Software Web Management User Interface. One attack included the deployment of an implant we called “BadCandy” which consisted of a configuration file (“cisco_service.conf”). The configuration file defined the new web server endpoint (URI path) used to interact with the implant. That endpoint receives certain parameters that allowed the actor to execute arbitrary commands at the system or IOS level.  

Another example is from September 2023, when CISA wrote about how BlackTech was observed modifying firmware to allow the installation of a modified bootloader which helps it to bypass certain security features (while creating a backdoor to the device). 

Detecting the modification of firmware is extremely difficult for defenders. Occasionally, there may be something in the logs to imply an upgrade and reboot, but turning off logging is usually one of the first steps attackers take once they are inside a network. 

This again highlights the need for organizations to sunset aging network infrastructure that isn’t secure by design, or, at the very least, increasing cybersecurity due diligence on older equipment such as configuration management. Performing configuration comparisons on firmware may help to highlight when it has been altered by an adversary.  

2) Uploading customized/weaponized firmware

If threat actors cannot modify the existing firmware, or they need additional levels of access that they don’t currently have, adversaries can upload customized or old firmware they know have working exploits against it (in effect, reverting to an older version of the firmware).  

Once the weaponized firmware has been uploaded, they reboot the device, and then exploit the vulnerability that is now unpatched. This now provides the threat actor with a box that can be modified with additional functionality, to exfiltrate data, for example. 

Again, as with the modification of firmware tactic, it’s important to check your network environment for unauthorized changes. These types of devices need to be watched very closely, as threat actors will want to try and prevent system administrators from seeing the activity by turning off logging. If you’re looking at your logs and it looks like someone has actually turned off logging, that is a huge red flag that your network has been infiltrated and potentially compromised. 

3) Bypassing or removing security measures

Talos has also seen threat actors take measures to remove anything blocking their access to fulfil their goals. If for example they want to exfiltrate data, but there’s an access control list (ACL) that blocks the actor from being able to access the host, they may modify the ACL or remove it from the interface. Or they may install operating software that knows to not apply ACLs against certain actor IP addresses, regardless of the configuration. 

Other security measures that APTs will attempt to subvert include disabling remote logging, adding user accounts with escalated privileges, and reconfiguring SNMP community strings. SNMP is often overlooked, so we recommend having good, complex community strings and upgrading to SNMPv3 where possible. Ensure that your management of SNMP is only permitted to be done from the inside and not from the outside.  

The BadCandy campaign is a good example of how an actor can remove certain security measures. The adversary was able to create miniature servers (virtualized computers) inside of compromised systems which created a base of operations for them. This allowed the threat actors to intercept and redirect traffic, as well as add and disable user accounts. This meant that even if the organization were to reboot the device and erase the active memory, the adversary would still have persistent accounts – effectively a consistent back door.  

Additional campaign objectives

In our original threat advisory, we also posted a non-exhaustive list of the type of activities Talos has observed threat actors take on network infrastructure devices. The point behind this is that threat actors are taking the type of steps that someone who wants to understand (and control) your environment.  

Examples we have observed include threat actors performing a “show config,” “show interface,” “show route,” “show arp table” and a “show CDP neighbor.” All these actions give the attackers a picture of a router’s perspective of the network, and an understanding of what foothold they have. Other campaign objectives include: 

• Creation of hub-and-spoke VPNs designed to allow the siphoning of targeted traffic from network segments of interest through the VPN. 
• The capture of network traffic for future retrieval, frequently limited to specific IPs or protocols. 
• The use of infrastructure devices to deliver attacks or maintain C2 in various campaigns. 

Read more about campaign objectives 

Recommendations

The first thing to say when it comes to recommendations is that if you are using network infrastructure that is end of life, out of support, and now has vulnerabilities that cannot be patched, now really is the time to replace those devices.  

To combat the threat of aging network infrastructure as a target, Cisco became a founding member of the Network Resilience Coalition. Along with other vendors in this space and key governmental partners, the group is focussed on threat research and recommendations for defenders. The initial report from the Network Resilience Coalition was published at the end of January 2024 and contains a broad set of recommendations for both consumers of networking devices and product vendors. 

Earlier this month, Cisco’s Head of Trust Office Matt Fussa wrote about how organizations should view these recommendations, and the overall threat that end-of-life network infrastructure poses on a national security level.  

The report from the Network Resilience Coalition contains an in depth set of recommendations for network infrastructure defense. Here is a brief summary: 

Key recommendations from the report for network product vendors include: 

• Align software development practices with the NIST Secure Software Development Framework (SSDF). 
• Provide clear and concise details on product “end-of-life,” including specific date ranges and details on what support levels to expect for each. 
• Separate critical security fixes for customers and not bundle those patches with new product features or functionality changes. 
• Get involved in the OpenEoX effort in OASIS, a cross-industry effort to standardize how end-of-life information is communicated and provide it in a machine-readable format. 

Purchasers of network products should: 

• Favor vendors that are aligned with the SSDF, provide you with clear end-of-life information, and provide you with separate critical security fixes. 
• Increase cybersecurity diligence (vulnerability scanning, configuration management) on older products that are outside of their support period. 
• Periodically ensure that product configuration is aligned with vendor recommendations, with increasing frequency as products age, and ensure implementation of timely updates and patches. 

As the report says, “These recommendations, if broadly implemented, would lead to a more secure and resilient global network infrastructure and help better protect the critical infrastructure that people rely on for their livelihood and well-being.”  

From a Talos perspective, we are keen to re-emphasize this point and help our customers transition from equipment that has become end-of-life. Using networking equipment that has been built with secure-by-design principles such as running secure boot, alongside having a robust configuration and patch management approach, is key to combatting these types of threats. Ensure that these devices are being watched very carefully for any configuration changes and patch them promptly whenever new vulnerabilities are discovered. 

Proactive threat hunting is also one of the ways that organizations can root out visibility gaps and hints of incursion. Look for things like VPN tunnels, or persistent connections that you don't recognise. This is the type of thing that will be left in an attack of this nature.  

And finally, the definition of post compromise means that the attacker had gained some form of credentials to get them to the place where they could then launch the exploit and get deeper access to the device.  

Our recommendations are to select complex passwords and community strings, use SNMPv3 or subsequent versions, utilize MFA where possible, and require encryption when configuring and monitoring devices. Finally, we recommend locking down and aggressively monitoring credential systems like TACACS+ and any jump hosts. 

Additional resources

State-sponsored campaigns target global network infrastructure (Talos blog)  

Network Resilience: Accelerating Efforts to Protect Critical Infrastructure (Cisco blog) 

Network Resilience Coalition: Full report 

Securing Network Infrastructure Devices (CISA) 

The VPN Filter catastrophe that wasn’t 

Cisco Trust Center: Network Resilience