February 18, 2013

Raising the Bar: AV-Testing

We’re proud to share that our product was recognized by AV-TEST as a top mobile security product. It is a testament to our approach of using data and advanced automated analysis to detect and block malware before it spreads. This offers comprehensive detection and equally important, protects people with the speed and efficiency that the traditional AV industry has lacked.

We hold ourselves and products to the highest standard, and we encourage the security industry to constantly raise the bar. The fact is, emerging technologies are a challenge to test and benchmark, and it takes time to build a sound methodology that is an accurate measure of the real-world threat environment. While AV-TEST is making great headway to this effect, we seek to elevate the current testing methodology to include additional standards that we hold ourselves to:

  • Prevalence: Mobile security testing companies do not have direct visibility into the threats in the wild and therefore it is difficult for them to devise assessments that reflect the real-world threat space. Some samples in the AV-TEST corpus are aging and not representative of threats affecting people today. AV-TEST has made headway on this, but the most accurate test would be one that gives more weight to threats that are affecting consumers in real-time, and those that are the most severe.
  • Severity: All threats are not created equal. Accurate threat detection differentiates the nature and severity of a threat and relays that knowledge to a user. Toll fraud is different in severity from an aggressive ad network. The most accurate test would understand and test for that distinction. Calling out a genuinely malicious app only for its inclusion of adware does little to educate the user on the true severity of the threat.
  • False Positives: What a security app doesn’t detect is just as important as what it does. Trawling may be the best way to net a huge catch of fish, but its a shame if it snags a dolphin. Similarly, a mobile security app that identifies everything would come up with many false positives and alarm the user. Under the current test, a mobile security app is only tested for false positives against the top apps on Google Play, not apps that are close to malware. A successful false positive test would seek out the correct balance of detection and editing that protects the end user.
  • Speed: The newest malware is the most potentially harmful to users. The most accurate test would assess the ability to detect the new malware and protect users as early as possible. We recognize the difficulty of designing this test, but feel that it is particularly important for users.

We will continue to hold our own work to the highest standards and we look forward to working with AV-TEST to ensure that their methodology provides a relevant and accurate measure of the real-world threat environment.

2 comments
  1. Mike says:

    I agree entirely with what you said, however I still applaud the efforts of AV-Test and others like it for at least giving some level of credibility assurance for the average user to look to. There are so many security suites in the Android market and you can’t rely on user ratings for those because all they are really rating is usability and not the security itself. People need a good frame of reference for what is actually going to protect them.

    Glad to see you guys scored well though, I have been a faithful user since my first HTC Incredible…. :)

Leave a comment