Showing posts with label SiteAdvisor. Show all posts

Malicious Web Crawling

June 27, 2006
SiteAdvisor indeed cashed for evaluating the maliciosness of the web, and New Zealand feels that nation wide google hacking initiatives are a more feasible solution to the problem of google hacking, compared to the Catawba County Schools Board of Education who blamed Google for indexing student test scores & social security numbers. It's like having a just-moved, 25/30 years old neighbors next to your place, who didn't know you have thermal movement detection equipment and parabolic microphones, in order to seal the house by using robots.txt, or assigning the necessary permissions on the web server asap.

Tip to the Board of Education, don't bother Google but take care of the problem on your own, immediately, through Google's automatic URL removal system, by first "inserting the appropriate meta tags into the page's HTML code. Doing this and submitting via the automatic URL removal system will cause a temporary, 180-day removal of these pages from the Google index, regardless of whether you remove the robots.txt file or meta tags after processing your request."

Going back to the idea of malicious web crawling, the best "what if" analysis comes from Michal Zalewski, back in 2001's Phrack issue article on "The Rise of the Robots" -- nice starting quote! It tries to emphasize that "Others - Internet workers - hundreds of never sleeping, endlessly browsing information crawlers, intelligent agents, search engines... They come to pick this information, and - unknowingly - to attack victims. You can stop one of them, but can't stop them all. You can find out what their orders are, but you can't guess what these orders will be tomorrow, hidden somewhere in the abyss of not yet explored cyberspace. Your private army, close at hand, picking orders you left for them on their way. You exploit them without having to compromise them. They do what they are designed for, and they do their best to accomplish it. Welcome to the new reality, where our A.I. machines can rise against us."

That's a far more serious security issue to keep an eye on, instead of Google's crawlers eating your web site for breakfast. Continue reading →

Consolidation, or Startups Popping out Like Mushrooms?

June 13, 2006
If technology is the enabler, and the hot commodity these days, spammers will definitely twist the concept of targeted marketing, while taking advantage of them. Last week I've mentioned the concepts of VoIP, WiFi and Cell phone spam that are slowly starting to take place.

Gartner recently expressed a (pricey) opinion on the upcoming consolidation of spam vendors, while I feel they totally ignored the technological revolution of spamming to come -- IPSec is also said to be dead by 2008..

"The current glut of anti-spam vendors is about to end, analysts at Gartner said Wednesday. But enterprises shouldn’t stay on the sidelines until the shakeout is over. By the end of the year, Gartner predicted, the current roster of about 40 vendors in the enterprise anti-spam filtering market will shrink to fewer than 10. As consolidation accelerates and as anti-spam technology continues to rapidly change, most of today’s vendors will be "left by the wayside," said Maurene Caplan Grey, a research director with Gartner, and one of two analysts who authored a recently-released report on the state of the anti-spam market."

The consequence of cheap hardware, HR on demand, angel investors falling from the sky on daily basis, and acquiring vendor licensed IP, would result in start ups popping up like mushrooms to cover the newly developed market segments, and some will stick it long enough not to get acquired given they realize they poses a core competency.

Sensor networks, spam traps, bayesian filters, all are holding the front, while we've getting used to "an acceptable level of spam", not the lack of it. What's emerging for the time being is the next logical stage, that's localized spam on native languages, and believe it or not, its gets through the filters, and impacts productivity, the major problem posed by spam.

SiteAdvisor -- I feel I'm almost acting as an evangelist of the idea -- recently responded to Scandoo's concept, by wisely starting to take advantage of their growing database, and provide the feature in email clients while protecting against phishing attacks. End users wouldn't consider insecure search by default in order to change their googling habits, they trust Google more than they would trust an extension, and they'd rather have to worry about Google abusing their click stream, compared to anything else. Anti-Phishing toolbars are a buzz, and it's nice to see the way they're orbiting around it.

Be a mushroom, don't look for an umbrella from day one! Continue reading →

Spotting valuable investments in the information security market

April 18, 2006
Back in January I mentioned the possible acqusition of SiteAdvisor in my "Look who's gonna cash for evaluating the maliciousness of the Web?" post and it seems McAfee have realized the potential of this social-networking powered concept on a wide scale, and recently acquired SiteAdvisor -- this was meant to happen one way or another and with risk of being over-enthusiastic I feel I successfully spotted this one.



Next to SiteAdvisor's pros and cons that I commented on, I also provided a resourceful overview of some of the current malware crawling projects out there, to recently find out that WebRoot finally went public with the Phileas spyware crawler, and that Microsoft's Strider Crawler came up with the Typo-Control project -- great idea as a matter of fact. What are some of the current/future trends in the information security industry? Are the recent flood of acquisitions the result of cheaper hardware and the utilization of open-source software, thus cutting costs to the minimum while the idea still makes it to the market?


Have both, entry and exit barriers totally vanished so that anyone could get aspired of becoming a vendor without the brand at the first place? Excluding the big picture, it is amazing how uninformed both, end and corporate users are, yet another lack of incentive for security vendors to reach another level of solutions -- if it ain't broken, don't improve it.



Moreover, what would the effect be of achieving the utopian 100% security on both, the market and the world's economy? On one hand we have "the worst year" of cybercrime, whereas spending and salaries are booming, and they should be as the not knowing how much security is enough, but trying to achieve the most secured state is a driving factor for decades to come.


The bottom line is, the more insecurities, the more security spending, the higher the spending, the higher the growth, and with increasing purchasing power, corporate R&D, and government initiatives you have a fully working economic model -- going to war, or seeing terrorists everywhere is today's driving force for military/intelligence spending compared to the "Reds are everywhere" propaganda from both camps of course, back in the Cold War period. Fighting with inspired bureaucrats is always an issue as well.



The Ansoff's Product/Market Matrix often acts as the de-facto standard for developing business opportunities, that is, of course, if you're not lead by a visionary aim, promote an internal "everyday startup" atmosphere to stimulate creativity, or benchmark against competitors. On the majority of occassions a security vendor is looking for ways to diversify its solutions' portfolio, thus taking advantage of re-introduced product life cycles and new sources for revenues.


While there should be nothing wrong with that given a vendor is actually providing a reliable solution and support with it, I often argue on how marketable propositions centric business model is not good for the long-term competitiveness of the company in question.



It's the judgement and competitors myopia that I'm talking about. In respect to the current information security market trends, or let's pick up the anti virus solutions segment, that means loosing sight of the big picture with the help of the mainstream media -- cross refferenced malware names, "yet another" malware in the wild, or supposed to be Russian hacker selling his soul for E-gold(cut the stereotypes here and go through the majority of recent statistics to see where all that phishing, spam and malware is coming from), is a common weakness of a possible decision-maker looking for acquisitions. Focusing on both, current trends, and current competitions is the myopia that would prevent you from sensing the emerging ones, the ones that would improve your competitiveness at any time of execution of course.



The way we have been witnessing an overal shift towards a services based world economy in comparisson to a goods based one, in the informaiton security market services or solutions will inevitably profiliate in the upcoming future. When was the last time you heart someone saying "I don't need an anti-virus scanner, but an anti-virus solution, what's yours and how is it differentiated from the others I'm aware of"? Un-informed decisions, quick and cheap way to get away with the "security problem", or being totally brainwashed by a vendor's salesforce would result in enormous long-term TCO(total cost of ownership) problems, given someone actually figures a way to make the connection in here.



Some time ago, I came across a great article at CSOOnline.com "2 Vendor Megatrends and What They Mean to You" giving insight on two trends, namely, consolidation of security providers and convergence -- the interception between IT and physical security. And while it's great in respect to covering these current trends, I feel the article hasn't mentioned the 3rd one - Diversification. An excerpt :



"One trend is consolidation. "We're seeing the bigger players buying out many of the smaller companies. And I think the largest of the security firms are looking to provide a full range of enterprise services," says C. Warren Axelrod, director of global information security at Pershing, a Bank of New York Securities Group company. "The larger firms, like Internet Security Systems, Symantec and Computer Associates, are buying in many areas to complement what they have. They're basically vying for control of the security space." Axelrod is dead on, and consolidation is just as rampant among physical security vendors as it is in the IT world."



I feel consolidation is happening mainly because different market segments are constantly getting crowded and mainly because it's very, very hard to get a name in the information security market these days, so instead of run for your own IPO, compete against market players whose minor modification may ruin your entire idea, you'd better get acquired one way or another. @stake is an example of how skilled HR runs away from the acquirer, at least for me counting the HR as the driving force besides the brand.



More from the article :

"The second trend is convergence—the confluence of IT and physical security systems and vendors—which, in some sense, is another form of consolidation, only it's happening across the line that historically divided those two worlds."



Tangible security is often favored by investors as it targets the masses, and the most visible example besides perimeter based defenses are the hardware appliances themselves. These days, there isn't a single anti virus, anti spam or anti spyware solution provider without a hardware appliance, but what's to note is how their OEM agreements are still working and fully applicable, it's all about greed, or let's avoid the cliche and say profit maximization -- whatever the market requires the vendors deliver!



Very in-depth article, while I can argue that vendors are so desperate to "consolidate bids" on a national level, as they usually try to get as big part of the pie as possible. What's else to note is that the higher the market transparency, the more competitive the environment, thus greater competition which is always useful for the final user. In respect to heterogenity and homogenity of security solutions, and all-in-one propositions, the trade-offs are plain simple, cut total TCO by using a single vendor, get your entire infrastructure breached into by an attacker that would sooner or later find a vulnerability in it -- find the balance and try to avoid the myth that complexity results in insecurities, as it's a unique situation every time.



What we're witnessing acquisition-to-solution turn-around periods of several months in response to an emerging market - the IM one, mobile anti-virus scanners seem to be the "next big thing", whereas it would take quite some time for this segment to develop, still you'd better be among the first to respond to the interest and the fact that there are more mobile phones capable of getting infected with a virus, than PCs out there -- 3G, 4G, mobile banking would fuel the growth even more, and these are just among the few issues to keep in mind. In a previous post, I also mentioned on a creative use of security intelligence information in Sophos's Zombie Alert service, and a product-line extensions, namely McAfee's bot killing system. What no one pictured would happen is emerging these days - vulnerabilities turning into IP and the overal commercialization of the security vulnerabilities market, and getting paid for getting hacked is a growing trend as well -- much more's to come for sure.



The secrets to successful acquisitions?

- retain the HR that came with it, and better put something on the table at the first place
- don't try to cannibalize the culture there, Flickr is the perfect example out of the security market
- go beyond the mainstream media sources, and PR releases, use open source competitive intelligence tools in order not to miss an opportunity
- attend as much cons as possible to keep track of who's who and where's the industry heading to
- cost-effectively keep in touch with researchers, and an eye on their blogs, you never know who would be your early warning system for business development ideas



Try to stay on the top of security, not in line with it.



Technorati tags:
, , , , , , , Continue reading →

Look who's gonna cash for evaluating the maliciousness of the Web?

February 14, 2006
Two days ago, SecurityFocus ran an article "Startup tries to spin a safer Web" introducing SiteAdvisor :

"A group of graduates from the Massachusetts Institute of Technology (MIT) aim to change that by crawling the Web with hundreds, and soon thousands, of virtual computers that detect which Web sites attempt to download software to a visitor's computer and whether giving out an e-mail address during registration can lead to an avalanche of spam.

The goal is to create a service that lets the average Internet user know what a Web site actually does with any information collected or what a download will do to a computer, Tom Pinckney, vice president of engineering and co-founder of the
start-up SiteAdvisor, said during a presentation at the CodeCon conference here."

The concept is simply amazing, and while it's been around for ages, it stills needs more acceptance from decision makers that tend to stereotype on perimeter and antivirus defense only. Let's start from the basics, it is my opinion that users do more surfing than downloading, that is, the Web and its insecurities represent a greater threat than users receiving malware in their mailboxes or IMs. And not that they don't receive any, but I see a major shift towards URL droppers, and while defacement groups are more than willing to share these with phishers etc., a URL dropper is easily getting replaced by an IP one, so you end up having infected PCs infecting others through hosting and distributing the malware, so sneaky, isn't it? My point is that initiatives such as crawling the web for malicious sites, listing, categorizing and updating their status is a great, both security, and business sound opportunity. The way you know the bad neighbourhoods around your town, in that very same way you need a visualization to assist in research, or act as a security measure, and while its hard to map the Web and keep it up to date, I find the idea great!

So what is SiteAdvisor up to? Another build-to-flip startup? I doubt so as I can almost feel the smell of quality entrepreneurship from MIT's graduates, of course, given they assign a CEO with business background :) APIs, plugins, already tested the majority of popular sites according to them, and it's for free, at least to the average Internet user who's virtual "word of mouth" will help this project get the scale and popularity necessary to see it licensed and included within current security solutions. They simply cannot test the entire Web, and I feel the shouldn't even set it as an objective, instead map the most trafficked web sites or do so on-the-fly with the top 20 results from Google. I wonder how are downloads tested, are they run through VirusTotal for instance, and how significant could a "push" approach from the end users, thus submitting direct links to malicious files found within to domain for automatic analysis, sound in here?

I think the usefulness of their idea could only be achieved with the cooperation/acquisition of a leading search engine, my point is that some of the project's downsizes are the lack of on-the-fly ability(that would be like v2.0 and a major breakthrough in respect to performance), how it's lacking the resources to catch up with Google on the known web (25,270,000,000 according to them recently), how IP droppers instead of URL based ones totally ruin the idea in real-life situations(it takes more efforts to register and maintain a domain, compared to using a zombie host's capabilities to do the same, doesn't it?)

In one of my previous posts on why you should aim higher than antivirus signatures protection only I mentioned some of my ideas on "Is client side sandboxing an alternative as well, could and would a customer agree to act as a sandbox compared to the current(if any!) contribution of forwarding a suspicious sample? Would v2.0 constitute of a collective automated web petrol in a PC's "spare time"?

Crawling for malicious content and making sense of the approaches used in order to provide an effective solutions is very exciting topic. As a matter of fact in one of my previous posts "What search engines know, or may find about us?" I mentioned about the existence of a project to mine the Web for terrorist sites dating back to 2001. And I'm curious on its progress in respect to the current threat of Cyberterrorism, I feel both, crawling for malicious content and terrorist propaganda have a lot in common. Find the bad neighbourhoods, and have your spiders do whatever you instruct them to do, but I still feel quality and in-depth overview would inevitably be sacrificed for automation.

What do you think is its potential of web crawling for malicious content, and by malicious I also include harmful in respect to Cyberterrorism PSYOPS (I once came across a comic PSYOPS worth reading!) techniques that I come across on a daily basis? Feel free to test any site you want, or browse through their catalogue as well.

You can also find more info on the topic, and alternative crawling solutions, projects and Cyberterrorism activities online here :

A Crawler-based Study of Spyware on the Web
Covert Crawling: A Wolf Among Lambs
IP cloaking and competitive intelligence/disinformation
Automated Web Patrol with HoneyMonkeys Finding Web Sites That Exploit Browser Vulnerabilities
The Strider HoneyMonkey Project
STRIDER : A Black-box, State-based Approach to Change and Configuration Management and Support
Webroot's Phileas Malware Crawler
Methoden und Verfahren zur Optimierung der Analyse von Netzstrukturen am Beispiel des AGN-Malware Crawlers (in German)

Jihad Online : Islamic Terrorists and the Internet
Right-wing Extremism on the Internet
Terrorist web sites courtesy of the SITE Institute
The HATE Directory November 2005 update (very rich content!)
Recruitment by Extremist Groups on the Internet

Technorati tags:
, , , , ,
Continue reading →