ransomware detection ransomware detection

Ransomware Detection That Actually Works: Behavioral, Network, and Deception Methods Compared

Mandiant’s M-Trends 2026 report puts the median dwell time for intrusions in 2025 at 14 days, up from 11 the year before. For ransomware specifically, the picture is uglier in a different way: median dwell time for ransomware-related intrusions was 6 days overall, five days for adversary-notified events, and 29 days for intrusions discovered internally. Translation — when ransomware crews are running the operation, they finish before defenders find them, and “discovery” is usually the ransom note.

That gap is what detection has to close. Signature-based antivirus flags known binary hashes and stops the threats it already knows about. Everything else — the unknown payloads, the abuse of legitimate tools, the human-operated intrusions where the encryption stage is the last thing that happens — requires detection methods built around behavior, network telemetry, or deliberately placed bait. Each of the three works on a different part of the kill chain. Each fails in different ways. This piece compares them on what they actually catch, what they miss, and where they belong in a stack.

Why Signature Detection Alone Stopped Being Enough

Signature engines compare files against hashes of known malicious binaries. This technique is effective in detecting known ransomware variants, but it is less effective against new or modified variants. Modern ransomware affiliates rotate builds constantly, run payloads in memory to avoid touching disk, and increasingly use what’s already on the box — PowerShell, WMI, scheduled tasks, legitimate remote-management tools — so there’s no novel binary to hash in the first place.

The bigger structural problem is timing. By the time a ransomware binary hits disk and a signature fires, the attacker has typically been in the environment for days. They’ve moved laterally, harvested credentials, exfiltrated data, and identified backup infrastructure. In 2022, the median time between initial access and handoff to a secondary threat group was over eight hours. Three years later, that window has collapsed to 22 seconds. A detection method that fires on the encryption payload is detecting the conclusion of the attack, not the attack itself.

Behavioral, network, and deception methods all exist to fire earlier. They differ in where they look and what they cost.

Behavioral Detection: Watching What the Process Does

Behavioral detection — usually delivered through Endpoint Detection and Response (EDR) — monitors what processes do rather than what they are. Monitoring system and user behaviours enables the detection of unusual activities, such as unexpected file encryption processes or unauthorized access attempts, indicative of ransomware presence. Behavioral detection is more effective against new and modified variants of ransomware, as it does not rely on known signatures.

The signals that matter for ransomware specifically:

  • Mass file modification with high entropy. A process opening hundreds of files per second and rewriting them with high-entropy contents is, mechanically, what encryption looks like. Most ransomware families trigger this signal because most ransomware actually encrypts files.
  • Volume Shadow Copy deletion. vssadmin delete shadows /all /quiet and equivalents are a near-universal precursor to encryption. Ransomware operators delete shadow copies so victims can’t roll back.
  • Boot configuration tampering. bcdedit /set {default} recoveryenabled No and related commands disable Windows recovery.
  • Suspicious child processes from Office applications. winword.exe spawning powershell.exe or cmd.exe is a high-fidelity indicator across the MITRE ATT&CK techniques T1059.001 (PowerShell) and T1204.002 (malicious file).
  • LSASS access from non-standard processes. Credential dumping (T1003.001) precedes lateral movement in nearly every human-operated ransomware case.
  • Privilege escalation followed by remote service creation. This is the lateral-movement signature of PsExec, WMI, and the SMB beacon patterns common to Cobalt Strike and its successors.

Vendor implementations layer machine learning on top of these primitives to reduce false positives and catch variations. Static and dynamic detection by signatures performed at various levels (through hashes at a file or ELF/PE section level) and file activity (write access to files with high entropy changes) has the advantage of being easily and rapidly implemented, but is also likely to generate false positives.

The strength of behavioral detection is coverage: it sees the process tree, the syscalls, the registry edits, the file operations. The weakness is that it sees them only on instrumented endpoints. Unmanaged devices, IoT, network appliances, ESXi hosts, and Linux servers without an agent are blind spots — and ransomware operators have been exploiting those blind spots aggressively. Targeting hypervisors directly to encrypt the underlying VM disks bypasses every Windows EDR agent in the guest operating systems. Behavioral detection also struggles with “living off the land” tradecraft, where the malicious process is powershell.exe or rundll32.exe and the bad behavior looks statistically similar to legitimate administration.

DETECTION METHOD COMPARISON
Where Each Method Fires in the Ransomware Kill Chain
BEHAVIORAL / EDR
Process & file activity
Watches syscalls, file ops, child processes, registry edits on each host.
Fires on:
Encryption activity, shadow-copy deletion, LSASS access, suspicious process trees.
Blind to:
Unmanaged hosts, ESXi, embedded devices, well-disguised LOLBin abuse.
NETWORK / NDR
Traffic & flows
Watches beaconing, DNS, SMB, RDP, exfiltration patterns at chokepoints.
Fires on:
C2 callbacks, lateral SMB/RDP, DGA domains, abnormal east-west flows.
Blind to:
Encrypted payloads without metadata anomalies, intra-host malice, on-keyboard activity.
DECEPTION
Decoys & canaries
Plants fake files, credentials, shares, hosts that legitimate users never touch.
Fires on:
Any access to a decoy. Near-zero false positives by design.
Blind to:
Targeted attacks that route around decoys; produces no signal until tripped.

Network Detection: Following the Wire

Network Detection and Response (NDR) tools watch traffic rather than hosts. They consume packet captures, NetFlow, Zeek logs, or SIEM-correlated network telemetry, looking for the patterns ransomware operations produce on the wire. In 2023, ransomware attacks that relied on C2 infrastructure made up 70% of all cyberattacks globally.

The signals here are different from behavioral detection but no less specific. Beaconing is the canonical one — a compromised host calling out to a command-and-control server at regular intervals. RITA and AC-Hunter’s strength lies in their ability to automatically identify these behavioral anomalies at scale, cutting through the noise to surface the handful of connections that warrant investigation. Modern operators tune their beacons to vary intervals and blend with legitimate background noise, but consistent low-volume callbacks to fresh infrastructure remain detectable through statistical analysis.

DNS tells another part of the story. Domain Generation Algorithms (DGAs), unusually high NXDOMAIN rates from a single host, DNS tunneling with abnormally large TXT or A record lookups, and queries to recently registered domains all point to malware infrastructure. DNS monitoring is also key for spotting algorithmically generated domains (DGAs) and catching strange query behavior.

The east-west traffic story is where ransomware-specific NDR earns its keep. Every major ransomware campaign requires lateral movement before payload deployment. Operators need domain admin or SYSTEM access on every target host, which requires moving to a domain controller via remote services. Without lateral movement, ransomware encrypts only the initial host. The detectable signatures of that movement are unusual SMB sessions between workstations, RDP from systems that don’t normally initiate it, WinRM from suspicious accounts, and Kerberos ticket activity consistent with pass-the-hash or Kerberoasting (T1550.002, T1558.003).

A 2025 Darktrace investigation of a SocGholish-to-RansomHub intrusion chain illustrates how this looks in practice: analysts identified systems connecting to destination ports such as 2308, 2311, 2313 and more — all on the same destination IP address associated with the RansomHub C2 environment. The attackers were also abusing WebDAV and NTLM over SMB to trigger authentication attempts. The attackers were also able to plant SCF files internally to expose NTLM hashes from users browsing shared folders. None of that is visible from a single endpoint. All of it is visible on the wire.

The weakness of network detection is parallel to behavioral’s strength: it can’t see what happens inside a single host. It also struggles with encrypted traffic — TLS 1.3 with encrypted SNI, DNS-over-HTTPS, and legitimate cloud services as C2 staging (Slack, Teams, GitHub) all degrade visibility. Modern C2 threat intelligence shows that threat actors increasingly hide behind Microsoft Teams, Slack, Google Workspace, and other trusted platforms rather than suspicious domains.

Deception: Bait That Only the Wrong People Touch

Deception flips the detection problem upside down. Instead of trying to identify malicious activity in a sea of legitimate activity, deception plants resources that legitimate users have no reason to access — so any access is, by construction, suspicious.

The category includes several distinct things. Honeypots are full decoy systems mimicking real ones. Honeyfiles or canary files are fake documents seeded in real directories. Honeytokens are fake credentials, API keys, AWS access keys, or document URLs that beacon when used. Canary tokens in the Thinkst sense are tripwires embedded in files, DNS records, or web links that fire on access.

The detection logic is unusually clean. Canary files are decoy files placed in sensitive directories or locations within a system. If an attacker attempts to access or modify these files, it triggers an alert, signaling a potential intrusion or unauthorized activity. There is no machine learning, no statistical baseline, no entropy threshold. The file shouldn’t be touched. It got touched. Alert.

Endpoint vendors have built canary files directly into their ransomware protection. Originating in Elastic 7.14, Elastic Endpoint ransomware protection uses canary files with the purpose of attempting to honeypot ransomware by (over-)writing some specific files. This provides a high confidence indicator that the culprit process is attempting to encrypt all files. A canary file acts and looks exactly like any other file — it can have valid content (DOCX, PDF, etc.), hidden, or marked as a system file to avoid user tampering. When ransomware indiscriminately walks a directory tree encrypting everything, it eventually hits the canary, and the EDR kills the process.

Network deception works the same way at the protocol level. Fake SMB shares advertised on a network with attractive names — Backups, Finance, Domain_Admin_Creds — sit untouched by legitimate users. A process scanning for shares to encrypt or accounts to compromise will find them. A real-world example: A security manager had used a Thinkst Canarytoken embedded in a Word Document as a honeyfile. The manager named the file passwords.docx and filled it with hundreds of fake username/password combinations to increase the size of the file and make it more attractive. One Saturday night, the manager received an email alert that the file had been opened, in Ukraine. The manager called the Security Operation Center (SOC) to ask whether they had detected any malicious activity on the network, but they hadn’t. Out of an abundance of caution, they activated the organization’s incident response (IR) company, which came onsite early Sunday morning. After a few hours of hunting, the IR team found evidence of a ransomware attack in progress.

The ransomware-specific value of deception is that the reported median dwell time was eight days. That’s the time between when an attacker accesses their victim’s systems and when the attack is either detected or executed. Combine that data point with another one — that attackers take only 16 hours to reach Active Directory once they have landed — and the takeaway is that threats go undetected for an average of seven days. Deception compresses that gap because attackers stumble into it during their reconnaissance, not at the encryption stage.

The honest weaknesses: deception is silent until tripped. It produces no signal during periods when nothing is happening, which makes it useless as a baseline detection layer. A targeted attacker who already knows the network can route around decoys. And deception only helps if someone is watching the alerts — the canarytoken story above hinged on a human seeing the email at night and calling the SOC.

How They Stack Together in a Real Detection Chain

RANSOMWARE KILL CHAIN
Detection Coverage by Phase
PHASE
BEHAVIORAL
NETWORK
DECEPTION
Initial access
Strong (phishing, exploit chains)
Moderate (exploit traffic)
Weak
C2 establishment
Moderate (process telemetry)
Strong (beaconing, DNS, DGAs)
Weak
Discovery / recon
Moderate (LOLBin abuse)
Moderate (port scans, AD queries)
Strong (decoys hit during enum)
Credential access
Strong (LSASS, Mimikatz)
Moderate (Kerberoasting traffic)
Strong (honeycredentials)
Lateral movement
Moderate (PsExec, WMI)
Strong (east-west SMB/RDP)
Strong (decoy hosts/shares)
Exfiltration
Moderate (rclone, archive tools)
Strong (volume, destination)
Moderate (canary docs leave)
Encryption / impact
Strong (entropy, mass writes)
Weak (already too late)
Strong (canary file write)

The pattern in the matrix is what matters. No single method covers the whole kill chain. Behavioral detection dominates the endpoint phases — credential access and encryption — but is weakest exactly where ransomware operators spend most of their dwell time, which is between hosts. Network detection inverts that profile, strongest on C2 and lateral movement, weakest on the final on-host actions. Deception spikes at discovery and credential-access phases where attackers actively probe the environment.

The Mandiant data on dwell time underscores why layering matters. Global median dwell time was 26 days when external entities notified, 5 days when adversaries notified (notably in ransomware cases), and 10 days when organizations discovered malicious activity internally. Internal discovery within 10 days requires catching something during the lateral-movement and discovery phases — exactly where network detection and deception are strongest. EDR alone tends to fire on either the initial implant or the encryption event, missing the long middle.

DWELL TIME REALITY · MANDIANT M-TRENDS
RANSOMWARE
6 days
Median dwell time for ransomware-related intrusions in 2024.
HAND-OFF SPEED
22 sec
Median time from initial access to handoff to a secondary group in 2025, down from 8+ hours in 2022.
EXTERNAL NOTIFICATION
25 days
Median dwell time when an outside party — not the victim — discovers the breach.

What Each Method Costs and What It Misses

Behavioral detection is the most expensive of the three on a per-endpoint basis and produces the highest volume of alerts. False positives on entropy-based ransomware detection are a known problem — legitimate backup software, disk encryption tools, and some compression utilities all do mass file writes with high-entropy output. Tuning is ongoing work, not a one-time exercise. The other behavioral pitfall is coverage gaps: ESXi hosts, network appliances, and IoT devices generally don’t run EDR, and ransomware operators have systematically targeted those gaps. Ransomware-related intrusions accounted for 13% of Mandiant investigations in 2025. Operators have moved beyond dual-threat encryption-and-theft operations toward systematically denying organizations the ability to recover, targeting identity services, virtualization management planes, and backup infrastructure.

Network detection cost scales with traffic volume and TLS decryption requirements. Decryption is contentious — it requires endpoint cooperation or a man-in-the-middle posture and creates privacy and compliance friction. Without it, encrypted C2 and exfiltration traffic looks like any other HTTPS session, and detection has to rely on metadata: connection timing, JA3/JA4 fingerprints, certificate anomalies, beacon intervals. The other limitation is intra-host visibility: a process talking to another process on the same machine produces no network traffic. East-west visibility also depends on where sensors sit. North-south chokepoints catch C2; only segment-to-segment sensors catch lateral movement.

Deception’s costs are the lowest of the three, and its false-positive rate is essentially zero by design. The hidden cost is operational discipline. Deception systems are great at detecting threats of all kinds, they can be the single best source of threat intelligence in your toolbox. But as powerful as they are, the thing they can’t do is respond to an event. When using deception systems of any kind, no matter how much attacker time they waste, if you aren’t closely monitoring them (and by close I mean 24×7) and responding when they alert, you will most likely find yourself with a compromised network and possibly some moderately useful forensic information after the event. A canary alert that arrives at 2 a.m. into an unmonitored mailbox is worthless. Deception also requires thoughtful placement — files in directories no one ever visits won’t be touched by ransomware that walks the file tree, and decoys with obvious naming patterns get flagged and avoided by sophisticated operators.

A subtler problem: deception coverage of Linux, container, and hypervisor environments is thinner than coverage of Windows. Tests on 53 Windows-based ransomware samples from seven families showed an ordinary detection time of about 12 s, often quicker than traditional methods like file hashing or entropy analysis. These detection results are currently limited to Windows-based ransomware environments, and do not yet cover Linux, containerized, or hypervisor-level ransomware.

Frequently Asked Questions

Does deception replace EDR? No. Deception fills gaps EDR can’t see — particularly on unmanaged hosts and during lateral movement — but produces no signal during normal operations and can’t enforce response. EDR remains the foundation; deception is force-multiplication.

Is signature-based AV obsolete for ransomware? Not obsolete, but insufficient. It still blocks high-volume commodity payloads cheaply. The problem is that human-operated ransomware affiliates rotate builds and use living-off-the-land techniques specifically to defeat it. Signatures are a baseline, not a strategy.

Where should an organization start if it has none of these? EDR with behavioral detection on every Windows and macOS endpoint, then network telemetry from internal chokepoints feeding either a SIEM or a dedicated NDR platform. Deception comes third because it requires a working SOC to act on alerts. The exception: free canarytokens take minutes to deploy and offer disproportionate early-warning value even with limited monitoring.

What about AI-powered detection? Most modern EDR and NDR vendors apply machine learning to behavioral baselining and anomaly scoring. The technology is real but oversold — ML reduces the noise floor on existing detection categories rather than inventing new ones. The MITRE ATT&CK techniques being detected don’t change because a neural network is doing the matching.

What Actually Works

The fastest detection times in real environments come from combining all three. Behavioral detection on every host that can run an agent. Network sensors at internal segment boundaries — not just the perimeter — to catch east-west movement. Canary files in attractive locations and canary credentials in places attackers will look during privilege escalation. Each method’s blind spot is another method’s strongest signal.

What doesn’t work is buying any one of them and treating ransomware detection as solved. The 2025 dwell-time numbers from Mandiant show defenders gaining ground on commodity intrusions and losing it on stealthier ones. Ransomware operators have spent the last three years optimizing specifically for the gaps between detection methods — agentless infrastructure, encrypted C2, hands-on-keyboard tradecraft that mimics admin behavior. The defenders who keep up are the ones who layer detection so that no single gap is fatal.

Add a comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Cybersecurity intelligence delivered directly to your inbox.

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Advertisement