When evaluating EDRs (Endpoint Detection and Response) solutions, it is essential to look beyond the telemetry they provide. While telemetry is important for creating detection rules and enabling in-depth investigations, an effective EDR solution requires a balance between several factors. Often, the best way to test the performance of an EDR is to perform scenario-based simulations in which we simulate a real threat actor. But this type of test is time-consuming and requires a great technical background. In this article, we will examine the various criterias that are essential when comparing EDR solutions.
Telemetry
Telemetry refers to the data collected from endpoints. High-quality telemetry allows for the creation of accurate detection rules and in-depth investigations. However, the focus should be on the relevance and comprehensiveness of the data rather than just the volume.
Key Points
- Comprehensiveness: ensure that telemetry covers a wide range of system events and activities. It should provide an accurate and holistic view of the state of the system and enable the identification of sophisticated threats that span multiple domains.
- Relevance: data should be pertinent to the security team.
- Granularity: the level of detail must enable in-depth forensic analysis. A high level of granularity enables the traceability of the exact steps in an attack. It is essential for understanding and mitigating complex incidents.
Comments
- Some EDR vendors choose not to consider certain telemetry for irrelevance or performance reasons
- Some telemetry is not transmitted to the administration console but is still used by the EDR agents locally
Detection Rules and Detection Engines
The quality of an EDR’s built-in engines and detection rules is crucial in identifying threats effectively. High-quality detection rules and engines are essential for detecting advanced and complex threats.
Key Points
- Accuracy: rules should precisely identify threats with minimal false positives.
- Adaptability: ability to update and refine rules as new threats emerge. An adaptable detection engine that can quickly incorporate new threat intelligence will stay relevant against evolving cyber threats.
- AI and Machine Learning: Use of advanced technologies to enhance detection capabilities. Machine learning models can identify patterns and anomalies that traditional signature-based methods might miss.
True Positive/False Positive Ratio
The true positive/false positive ratio of and EDR solution impacts the efficiency and productivity of a security team. An high false positive rate increases alert fatigue and unnecessary interventions.
Key Points
- High True Positive Rate: ensures threats are accurately identified. An EDR with a high true positive rate can effectively protect against real threats.
- High False Positive Rate: increases alert fatigue and unnecessary interventions. For example, an EDR that frequently flags legitimate activities as threats can overwhelm security teams and cause critical alerts to be missed.
Comments
- Some EDR vendors disable some detections capabilities of their engines in order to reduce the false positive rate. Each vendor has its own philosophy on the question.
- A favorable true positive/false positive ensures that security teams can trust the alerts they receive from the EDR
Remediation Capabilities
Remediation capabilities determine how well an EDR can respond to detected threats. Effective remediation capabilities can significantly reduce the impact of an incident. Automation combined with manual options provides a balanced approach to threat response.
Key Points
- Automated Responses: ability to automatically quarantine or neutralise threats. For example, an EDR that can automatically isolate a compromised system from the network, preventing the lateralization of malware.
- Manual Controls: options for security teams to respond to and manage incidents manually. Experts can be involved if required, providing flexibility in the response strategy.
- Scope of action: support for various remediation actions like file restoration, system rollback, threat isolation and more.
Exclusion accuracy
Exclusions allow fine-tuning of the EDR’s behavior. An EDR that supports precise exclusions can differentiate between legitimate business applications and malicious activity, reducing false positives while maintaining robust security.
Key Points
- Granularity : allows security teams to specify exact files, processes, signature or behaviors to exclude, reducing unnecessary alerts.
- Balanced Security: ensures that exclusion features do not create too large blind spots when activated.
Self-Sufficiency (when offline)
The self-sufficiency of an EDR agent when offline is crucial for continuous protection. Autonomous agents ensure that endpoints remain protected at all times, even in offline environments. This feature is essential for organisations with remote or mobile workforces.
Key Points
- Local Signature Database: presence of a local database to detect threats without internet connectivity.
- Local Detection Engines: ability to run detection engines and apply rules locally. This allows for real-time threat detection even when the central console is unreachable.
- Offline Remediation: capability to remediate threats even without a connection to the central console. For instance, the agent should be able to isolate threats and restore affected files autonomously.
Detection Speed
The detection speed is essential to prevent threats from causing significant damage. The faster the EDR is able to identify and mitigate a threat, the less impact it will have on the organisation’s network.
Key Points
- Real-Time Detection: ability to identify and respond to threats instantly. Immediate detection can stop malware before it spreads or causes serious harm.
- Minimal Latency: ensuring that detection processes do not lag and allow threats to execute. Low latency is crucial for detecting fast-moving attacks like ransomware.
Customization Options
Customization options allow an EDR to be tailored to specific organizational needs.
Key Points
- Rule Customization: ability to create and modify detection rules. For example, a company may want to create specific rules for its unique network architecture or business processes.
- Policy Management: flexibility in defining security policies and configurations. Custom policies can help align the EDR’s operations with the organization’s risk management strategy.
- Integration Capabilities: support for integrating with other security tools and systems. Seamless integration can enhance the overall security posture by enabling coordinated responses across different security solutions.
Reliability
Reliability ensures that the EDR can provide consistent protection without gaps, maintaining the security posture even during system failures or maintenance periods.
Key Points
- Availability : an EDR with high uptime ensures that endpoints are consistently monitored and protected.
- Self Repair : ability to automatically identify and fix issues that could compromise its functioning. This capability ensures that the EDR remains operational and effective, even in the event of system disruptions.
Interoperability
Interoperability measures the ability of an EDR to operate within the system without causing conflicts.
Key Points
- Compatibility: works with different operating systems. An EDR that supports multiple platforms can provide protection multiple environments.
- Minimal Disruption: ensures that the EDR does not interfere with system performance or other software. For example, an EDR should not cause system slowdowns or crashes, which can reduce productivity.
- Ease of Integration: simple integration with existing IT infrastructure. This includes compatibility with other security tools, ensuring a unified security ecosystem.
Self Protection
Self protection measures determine how resilient the EDR is against tampering and evasion. Strong integrity and bypass protection ensure that the EDR remains effective against attempts to disable or evade its detection capabilities, maintaining continuous and reliable protection.
Key Points
- Tamper Resistance: strong mechanisms to prevent unauthorized modifications (integrity checks, access rights, anti-tamper feature)
- Evasion Detection: ability to detect and counteract techniques used to bypass or disable the EDR
- Security Hardening: robust security architecture to resist attacks on its environment. This includes measures such as using secure communication channels or checking firewall rules.
Additional Criteria
Support and Documentation
Quality support and comprehensive documentation are crucial for effective use and maintenance of the EDR.
Cost and Licensing
Considering cost and licensing ensures that the EDR solution is sustainable and provides good value, aligning with the organization’s financial and operational needs.
User Interface and Usability
It might be overlooked but a user-friendly interface can significantly reduce the learning curve and improve the efficiency of security teams. Customizable dashboards and reports allow teams to focus on the most critical information and highlight relevant metrics. Clear and informative visualizations can help in quickly understanding and responding to security incidents
Tools
MITRE Engenuity ATT&CK Evaluations
Through the lens of the MITRE ATT&CK knowledge base, cybersecurity solutions were evaluated against diverse adversary simulations (based on APT’s behaviour).
EDR-Telemetry Project
This repo provides the telemetry retreived by each EDR product and other endpoint agents (non-exhaustive as not validated by the vendors)
Gartner Peer Insights
Gartner Peer Insights provides reviews and ratings from actual users of EDR products (based on the subjective opinions of end users)
Forrester Wave
Forrester Research evaluates and ranks EDR vendors based on their current offering, strategy, and market presence.
Conclusion
The evaluation of EDR solution requires a broad approach that goes beyond the simple quality of the telemetry and includes factors such as the quality of the detection engines, the remediation capability or even the quantity of false positive alerts. It is essential that the vendor’s philosophy and technical choices match the needs and requirements of your organisation.
I hope this article will help you choose your next generation EDR solution. Please feel free to share with us in the comments any other tools or criteria that you think might help people compare EDR solutions.