| Main Archive Page > Month Archives > websecurity archives |
Re: [WEB SECURITY] scanner reviews, testing news
From: Arian J. Evans <arian.evans_at_nospam>Date: Tue Oct 02 2007 - 02:33:35 GMT
To: "Andre Gironda" <andreg@gmail.com>
Well this is a good reply, and my positions haven't changed
and should be well known, but I'll add some inline commentary:
On 10/1/07, Andre Gironda <andreg@gmail.com> wrote:
> Arian, Thanks for the discussion - please see comments inline
>
> On 9/28/07, Arian J. Evans <arian.evans@anachronic.com> wrote:
> > Has anyone done a "scanner review" of all scanners lately?
> > If so would someone point me at it. Thanks.
>
> Justin Schuh did one for Neohapsis while he was there, but he left Neo
> and his work was never released to the public. I think this fact is
> mentioned in this article (I suggest downloading the whole PDF as I
> think it's in there):
> http://www.networkcomputing.com/showArticle.jhtml?articleID=199000936&pgno=5
I don't think I've read that one. Thanks, I'll check it out.
> The other companies that are similar to Neohapsis are also likely to
> have an internal reviews, as do competitors around the space. I hate
> to call them all out, but Accuvant, Coalfire, Mandiant, Trustwave,
> Casaba, IOActive, Korelogic, Protiviti, and QuietMove all seem to
> stick out on the various PCI (ASV or QSA lists), WASC, and OWASP
> related material and email threads that I see.
Most VAR type reviews I've seen simply compare scanner output signal-to-noise ratios. At least one or two of those you listed completely reject scanners (based upon historical limited value)
> If you're a web application vulnerability scanning company and you
> don't know who your competitors are then I'm not going to bother with
> that list. In fact... what exactly do you mean when you say, "all
> scanners"? What specifically are you referring to? Or who?
I had four in mind, but was more interested in what repondents thought.
> > What is the link to the one done maybe last year that I think
> > some EU, possibly German, folks did?
>
> Never saw that. Do you remember any other details besides EU/German?
Others responded with the right review. Cenzic translated it.
> > Had such a nice call today. Someone asked me "so how
> > do you automatically scan and compare business logic
> > if you have different users/roles with different logic paths?"
>
> I was reading Felix Linder's, "Attack Surface of Modern Applications"
> talk from the recent HITB conference -
> http://conference.hackinthebox.org/hitbsecconf2007kl/?page_id=130 - of
> which the full presentation material is available here -
> http://conference.hitb.org/hitbsecconf2007kl/materials/D2T1%20-%20Felix%20Lindner%20-%20%20%09%20Attack%20Surface%20of%20Modern%20Applications.pdf
Thanks. This also looks very nice, and I haven't seen it.
> In this presentation, using MITRE CVE data, Felix shows that business
> logic flaws only compromise 4% of the probable vulnerability
> classifications in recent statistics. However, nearly 47% are left
> unclassified, meaning what I'm not exactly sure.
CVE isn't meant to handle this data. That's why MITRE has their CWE project:
[...]
> > Jeremiah's paper on business logic examples is great for
> > helping folks think about many of the *real* problems they
> > probably aren't finding with their scanners:
> >
> > http://www.whitehatsec.com/home/assets/WP_bizlogic092407.pdf
>
> You really think so? Why? I didn't get anything out of this paper.
> What is the problem it is trying to solve? Who is the target
> audience? I don't understand the examples and how they relate to the
> logical flaws in web applications. What part do you think will help
> folks understand semantic vs. logical flaws?
Interesting feedback. Always room to improve. It seems pretty clear to me that these are contextual vulns related to a business decision.
Now -- it's not clear was is design and pattern versus implementation and practice, but the doc wasn't for developers.
It's a clear case of common business rules gone awry, that can be used to abuse applications, but not >Click>Scan(ed) for.
> > Now I know some folks try to spin some of those things into
> > "parameter tampering" but I think we all know they are not
> > getting found by automation, and a few of those are especially
> > tricky to find in source without some meta-info about the app.
>
> There is nothing wrong with doing code review or fault-injection along
> with a dynamic/hybrid analysis assessment. Whether this is assisted
> by tools that automate part of the work or not is relative. Anything
> can be `automated' to some degree, even if it means that only workflow
> is improved or maybe the presentation of data is optimized for the
> human eye.
Sure. I didn't say anything was wrong with it.
This is what most folks today *are not* doing, and where I am seeing the majority of people fail.
> People think that web application security scanners are
> point-and-click. Are they? Do you really think that web application
> vulnerabilities can be fixed like OS or fat application
> vulnerabilities following a Qualys/Nessus/Foundstone scan?
Sure they can. It depends what type they are, and what the software/framework/etc. is. I personally have successfully been helping folks do this for a long time, so I am confident that this is the case.
> Scan & patch? I assume your answers from
> http://www.webappsec.org/lists/websecurity/archive/2007-10/msg00001.html
> are still the same, no?
Whatever we *could* do, and whatever the future holds, the pragmatic now is scan and patch.
> Last I heard, scanners of all types are only as good as the reviewers.
I do not disagree.
> Doesn't it require some knowledge of how the vulnerabilities work in
> order to weed out non-exploitables, or more interestingly - turn a
> false positive into a working exploit?
Yes it would.
> Wouldn't web application security scanners also require remediation
> steps/support? How is that automated?
I'm not sure how others do it. We can integrate with things like bug tracking systems. When a dev updates/closes a bug, you could ostensibly automatically retest just that thing, though most folks explicitly fire off a unit test for a section of code today.
> Can logical flaws be found through automated testing? I think that
> depends on what you mean by `automated testing'.
Completely automated scanning. The answer is: a small percentage, depending on your false-positive tolerance to wade through.
> In the case of parameter tampering "business logic" flaws, I would
> tend to think these could be found by running two scanners with
> different credentials twice. After scraping once for data posted, you
> could then compare a diff of the data to the second run. You could
> also use data from previous versions of the website - or possibly
> other website data you have. Even CSRF could be exposed in this way,
> such as seen by CSRF Dorks.
Sure. Credential diffs and deltas are useful, but they also generate a lot of noise most folks don't know how to evaluate. Per CSRF, it's the same thing. The "web" is CSRF.
It's where you are making a security decision, or authorization assumption for a sensitive transaction, that you probably care. Hard to scan for.
[...] good Imperva references [...]
> As you can see by their numbers, parameter tampering is the #1
> classification for vulnerabilities in web applications. Furthermore,
> they have even separate cookie poisoning into a separate category. If
> you read this section -
> http://www.imperva.com/application_defense_center/glossary/cookie_poisoning.html,
> they even claim, "Cookie poisoning is in fact a parameter tampering
> attack, where the parameters are stored in a cookie".
>
> What are your definitions for parameter tampering and cookie
> poisoning? How do they fit into logical flaws from your perspective?
A cookie is simply a name=value pair. I don't see "cookie poisoning" as different, in terms of param manipulation.
XSS or SQL Injection, that is a different thing, cookie NV-pair or not-cookie NV pair. Pretty simple.
> After reading Jeremiah's paper - I didn't feel as if I knew what the
> WhiteHatSec definition of business logic really means. Can you or he
> explain that further?
Notes above.
> If you want to know how to benchmark properly, this is a separate
> topic - so I feel we should make it a separate thread or that you
> should contact me off-thread about it.
In fact, benchmarking is one of the dearest subjects to my heart.
There is other info out there, and I was curious whom had read it and would point me at it. No one did, which is what I wanted to know.
Thanks again. -- Arian Evans software security stuff ---------------------------------------------------------------------------- Join us on IRC: irc.freenode.net #webappsec Have a question? Search The Web Security Mailing List Archives: http://www.webappsec.org/lists/websecurity/ Subscribe via RSS: http://www.webappsec.org/rss/websecurity.rss [RSS Feed]