| Main Archive Page > Month Archives > websecurity archives |
Re: [WEB SECURITY] High-level questions on Web Application Scanning
From: Arian J. Evans <arian.evans_at_nospam>Date: Sun Feb 24 2008 - 00:23:13 GMT
To: bikramkgupta@gmail.com, websecurity@webappsec.org
Bikram -- these are all great questions. You've walked into a tough challenge!
On Sat, Feb 23, 2008 at 9:51 AM, Bikram Gupta <bikramkgupta@gmail.com> wrote:
>
> I am gradually getting to work on Web application security and it is just
> difficult. Please bear if the questions sound stupid.
>
> 1) What is the common trend that you see in the positioning of a web
> application security team in the IT deployments? Does it necessarily have to
> be a separate team, or it's often combined with network/system security team?
This varies wildly depending on the organization. I work with many software startups, and even midsize software companies that do not have a "webappsec" person at all, or are just starting to add one.
I work with other groups, say online trading companies, that have multiple members and continue to add more. They are usually under the CSO and part of the security team. Those resources tend not work work well tied to building teams (aka development) or care-and-feeding teams (network security/operations). Not to mention network folks often struggle to understand this subject.
Software security is kind of a bastard security role. You need to understand software design and development, you need to have someone who understands testing software, and then you have your traditional information security needs (policy and business alignment, etc.). Network security folks rarely handle this appropriately.
> 2) On a scale of comparison, how much effort does a web application scanner
> (say, OWASP developed scanner) needs in comparison to a network/host scanner (Nessus)?
It's a very different task. Nessus is simply trying to find known vulnerabilities in known code by fingerprinting (in the vast majority of cases). Runtime (blackbox) and source code (whitebox) scanners are trying to find systemic weaknesses and exploitable vulnerabilities in unknown code. That's a tough task.
> 3) In your point of view, when would it make sense to have a combined
> scanner that does both network/host scanning and web scanning together?
Maybe. A CSO wants all their vuln data in one place.
However, software security issues should be valueable to the business owner who is unlikely to care about the network stuff, which in most cases to them boils down to a firewall or patch-management issue.
> Why didn't they build a web app scanner into Nessus?
Too much work. Too little understanding. (my opinion)
That's why you won't find an open source scanner that works.
Heck, most commercial scanners don't work very well yet.
> Is it because the job
> functions in web scanning Vs network scanning are just too mutually exclusive?
>
> 4) I have been reading through most of the scanner comparisons. One of these
> (http://www.virtualforge.de/whitepapers/web_scanner_benchmark.pdf) concluded
> that, "The best scanner found 13 out of 85 technical vulnerabilities
> (15.3%)". Now, from your experience, what is the statistical success rate
> that you see among the leading web app scanners? And what is normally done
> about the rest - unfound vulnerabilities?
There are a lot of stats bandied about, but the truth is, all software can be different and the degree to which a scanner of any kind is effective depends *a lot* on the software itself.
Sometimes scanners work really well, and sometimes they don't work well at all.
Most people are just learning how to find their vulns, and most are not aware of the vulns they aren't finding, let alone the ones they aren't fixing.
There are progressive companies that started doing this seven to eight years ago, but they use armies of expensive consultants to test their software.
A lot of people go through this learning curve where they start out with a scanner (and too few humans) and generate a lot of noise and crap they get excited about. 500 page reports. Maybe they purchase a WAF too.
A year later and they are starting to get overwhelmed and cannot deal with the false positives, and they starting trying to figure out how to act on all the data they have.
Two years later they've been exploited, or had large holes in their software disclosed to them that their scanner didn't find for whatever reason, and they realize that they have false-negatives to deal with too.
Quite a few folks I've talked to in the last year are hitting this phase and looking around for new approaches.
I believe you need a team of highly trained humans, and that you have augment them with the best software automation you can. I think the most effective thing you can do is black box testing combined with targeted source code testing.
Once you know your issues, then you can
strategize about how to deal with them.
There are different beliefs about how to approach this, and the debate gets religious. Just keep in mind that very few folks have much real-world experience in actually *solving* application security problems.
And virtually none of them, vendors or otherwise, have any real math to back up their claims or approaches.
The scanners themselves vary widely in quality so measure carefully.
So -- what do you do about all of this?
One size doesn't fit all. Try some different approaches and see what works in your organization
Threat Modeling catches things up front:
Some folks get a lot of mileage out of threat modeling. I'm a big fan myself. Others find it non-actionable and find they just need hard vuln data. You can do this yourself or I can recommend some consultants.
Black box testing w/automation scales nicely:
I know folks with dozens to hundreds of websites that change frequently, and there is no way they can do source review on more than a few of them in a useful manner, so they focus on scalable black box testing leveraging automation.
Source code analysis definitely has a place:
I also know people who have a few sites that do not change often, have rigorous change control, and utilizing frameworks with security controls combined with source code review seems to works well for them.
I know folks implementing WAFs for blocking issues they can't fix. This seems sound.
I also know folks throwing their WAFs away because they don't have the resources to keep up with configuring them. Most of the WAF vendors still sell them as "magic bandaids" and people get upset once they figure out that they aren't. By default the WAFs are pretty limited at protecting certain vulnerabilities without some context.
Disclaimer:
By the way, I do work for a vendor -- WhiteHat Security, which primarily focuses on black box testing using scanning technology and human eyeballs.
Prior to that, though, I performed pen testing
and source code reviews on applications for
many years, written testing methodologies
for everything from web apps to databases,
worked with both OWASP folks and WASC
folks. Heck I wrote web apps before HTML
3.2 was a standard, lol, so hopefully my
opinions here are not completely uninformed.
Cheers, -- Arian Evans software security stuff ---------------------------------------------------------------------------- Join us on IRC: irc.freenode.net #webappsec Have a question? Search The Web Security Mailing List Archives: http://www.webappsec.org/lists/websecurity/ Subscribe via RSS: http://www.webappsec.org/rss/websecurity.rss [RSS Feed]