Malicious Web Crawling

SiteAdvisor indeed cashed for evaluating the maliciosness of the web, and New Zealand feels that nation wide google hacking initiatives are a more feasible solution to the problem of google hacking, compared to the Catawba County Schools Board of Education who blamed Google for indexing student test scores & social security numbers. It's like having a just-moved, 25/30 years old neighbors next to your place, who didn't know you have thermal movement detection equipment and parabolic microphones, in order to seal the house by using robots.txt, or assigning the necessary permissions on the web server asap.

Tip to the Board of Education, don't bother Google but take care of the problem on your own, immediately, through Google's automatic URL removal system, by first "inserting the appropriate meta tags into the page's HTML code. Doing this and submitting via the automatic URL removal system will cause a temporary, 180-day removal of these pages from the Google index, regardless of whether you remove the robots.txt file or meta tags after processing your request."

Going back to the idea of malicious web crawling, the best "what if" analysis comes from Michal Zalewski, back in 2001's Phrack issue article on "The Rise of the Robots" -- nice starting quote! It tries to emphasize that "Others - Internet workers - hundreds of never sleeping, endlessly browsing information crawlers, intelligent agents, search engines... They come to pick this information, and - unknowingly - to attack victims. You can stop one of them, but can't stop them all. You can find out what their orders are, but you can't guess what these orders will be tomorrow, hidden somewhere in the abyss of not yet explored cyberspace. Your private army, close at hand, picking orders you left for them on their way. You exploit them without having to compromise them. They do what they are designed for, and they do their best to accomplish it. Welcome to the new reality, where our A.I. machines can rise against us."

That's a far more serious security issue to keep an eye on, instead of Google's crawlers eating your web site for breakfast. Continue reading →

Nation Wide Google Hacking Initiative

The idea of doing reconnaissance for the purpose of pen testing or malicious activity through google hacking, has already reached levels of automation -- the problem is how the threat gets often neglected by those that actually suffer from a breach later on. I came across to an article pointing out that :



"Anyone who wants to hack into sensitive information on New Zealand internet sites might be pleased to know it can be as easy as typing keywords into a Google search. Researchers at Massey University’s Albany campus say the country’s websites are more vulnerable to "Google hacking" than anywhere else in the world. University Information and Mathematical Sciences Institute senior lecturer Dr Ellen Rose and graduate student Natalia Nehring recently completed a study into the topic."



Not exactly a type of cyberterrorism exercise such as the most recent DigitalStorm, but it's logical to conclude that if someone takes the time and effort to data mine the web, localize the attack like in this case, a lot will be revealed. In a recent article, CSOonline goes in-depth into the security implications posed by Google. I once had a chat with Johnny Long on many topics, among the "few", of course, was google hacking. He made a good point on saying that it's whatever you actually do with the results that matters most, and how diverse is the threat -- by googling your lights off for instance.


What you should keep in mind is that it isn't Google to blame, the way "Improving the Security of Your Site by Breaking Into it" provoked awareness, and not damage. Think the problem isn't big of a shot -- gather some intelligence by yourself through the Google Hack Honeypot project. Continue reading →

Snooping on Historical Click Streams

In a previous post "The Feds, Google, MSN's reaction, and how you got "bigbrothered"? I gave practical advices on how can easily do your homework on the popularity of certain search terms and sites, without the need of issuing a subpoena. The other day, AlltheWeb (Yahoo!) introduced their Livesearch feature, seems nice, still it basically clusters possible opportunities. Now the interesting part, on the next day Google launched Google Trends which is :



"builds on the idea behind the Google Zeitgeist, allowing you to sort through several years of Google search queries from around the world to get a general idea of everything from user preferences on ice-cream flavors to the relative popularity of politicians in their respective cities or countries."

This is what I've been waiting for quite some time, and you can easily make very good judgements on key topics based on regions, languages, even cities -- marketers get yourself down to business!



Antivirus, Malware, Spyware, NSA, Censorship, Privacy



What's next, the rise of MyWare and its integration on the Web? Give a try to Yahoo!'s Buzz, and PacketStormSecurity's instant StormWatch as well. Continue reading →

The "threat" by Google Earth has just vanished in the air

Or has it actually? In one of my previous posts "Security quotes : a FSB (successor to the KGB) analyst on Google Earth" I mentioned the usefulness of Google Earth by the general public, and the possibility to assist terrorists. The most popular argument on how useless the publicly available satellite imagery is that it doesn't provide a high-resolution images, and recent data as well -- that's of course unless you don't request one, but isn't it bothering you that here we have a street-side drive-by POC?



The recently introduced Windows Live Local Street-Side Drive-by (A9's maps have been around for quite a while), is setting a new benchmark for interactive OSINT -- if any as this is also a privacy violation that can be compared with efforts like these if it was in real-time. Having had several conversations with a friend that's way too much into satellite imagery than me, I've realized that starting from the basic fact of targeting a well known or a movie-plot location doesn't really requires satellite imagery. I find that today's sources basically provoke the imagination and the self-confidence -- and hopefully nothing more!


There have been numerous articles on the threat posed by Google Earth, and India seems to be the most concerned country about this for the time being :



"Chief of the Indian Army General J.J. Singh warns that Google Earth could endanger national security by providing high resolution photographs of strategic defense facilities. The software could prove especially useful to countries that do not have their own satellite capabilities. Singh called Google Earth a shared concern for all countries, requiring all countries to cooperate to address the issue. Indian President APJ Abdul Kalam has also expressed concerns over Google Earth and national security."



You can spend hours counting the cars in front of NSA's parking lot through public satellite imagery resources, still you would never get to see what's going on in there, I guess things have greatly changed since the days when tourists sent over the USSR, or exactly the opposite, to the U.S, would try to get hold of as many maps as possible finish the puzzle.



In some of my previous posts on Cyberterrorism, I said that terrorists are not rocket scientists until we make them feel so, and I'm still sticking to this statement, what about you? As a matter of fact, Schneier is inviting everyone to participate in the Movie-Plot Threat contest -- stuff like terrorist EMP warfare, Nuclear truck bombs (the same story from 3 years ago), and other science fiction scenarios worth keeping an eye on.



Terrorism is a profitable paranoia these days, that's constantly fuelling further growth in defense and intelligence spending, as satellite imagery is promoted for the bust of Bin Laden, whereas their infrastructure seems to pretty safe, isn't it? (More photos, 1, 2, 3, 4, 5, 6) I'd rather we have known parties as an adversary, the way it used to be during the Cold War, whose competition sent us in Space, and landed us on the Moon , instead of seeing terrorists everywhere and missing the big opportunity.



Technorati tags:
Security, Terrorism, Cyberterrorism, Privacy, Google Earth, Google Maps, Space, Microsoft Live, New Media Continue reading →

Security vs Privacy or what's left from it

My latest privacy related posts had to do with "The Future of Privacy = don't over-empower the watchers!" and "Data mining, terrorism and security" in respect to the the still active TIA and the hopes for the effectiveness out of data mining. While these are important topics I feel every decent citizen living in the 21st century should be aware of -- many still "think conspiracies" than real-life scenarios. At the bottom line, privacy violations for the sake of your security and civil liberties are a common event these days!



Today, I came across an article "Google must capitulate to DoJ, says judge" in relation to the DoJ's subpoena trying to get access to random sites and searches in order to justify its statement that anti-porn filters do not protect young children online.


The NYtimes is also a running a story on this. What I truly liked is US District Judge James Ware's comment that he was reluctant to give the Justice Department everything it wanted because of the "perception by the public that this is subject to government scrutiny" when they type search terms into Google.com, that's right, but you would be also right to conclude that such requests would turn into a habit given Google's data aggregation power. It's s a complex process to run the world's most popular search engine when everyone wants to take a bite from you, at least they have hell of motto to sort of guide them in future situations like this, but is it?



This time it's a misjudged online porn request that gets approved, next time, it would be Google against the terrorists, again, for the sake of your Security, one backed up by a little bit of glue as on the majority of occasions!



Technorati tags :
Privacy, Google, Search Engine Continue reading →

Chinese Internet Censorship efforts and the outbreak

In some of my January's Security Streams, I did some extensive blogging expressing my point of view on the current Internet censorship activities, and tried to emphasize on the country whose Internet population is about to outpace the U.S one - China. In my posts "China - the biggest black spot on the Internet’s map", "2006 = 1984?", "Twisted Reality", you can quickly update yourself on some of the recent developments related to the topic, but what has changed ever since?


Government bodies such as the DoJ seem to favour the amount of data the most popular and advanced search engine Google holds and tried to obtain information for the purpose of "social responsibility". What's more to consider are some of the weak statements made, namely :



"House Government Reform Committee Chairman Tom Davis (R-VA) has criticized Google for refusing to hand search records over to the US Justice Department while cooperating with China in censoring certain topics. Justice sought the records to bolster its case against a challenge to online anti-pornography laws, but Google refuses to submit the records on privacy grounds. Davis does not expect a standoff between Google and the government, but hopes an agreement can be reached, allowing Google to supply the records without frightening users that their searches may be examined."



and in case you're interested, some of my comments, :



"Is it just me or that must be sort of a black humour political blackmail given the situation?! First, and most of all, the idea of using search engines to bolster the online anti-pornography laws created enough debate for years of commentaries and news stories, and was wrong from the very beginning. Even if Google provide the data requested it doesn’t necessarily solve the problem, so instead of blowing the whistle without any point, sample the top 100 portals and see how they enforce these policies, if they do. As far as China is concerned, or actually used as a point of discussion, remember the different between modern communism, and democracy as a concept, the first is an excuse for the second, still, I feel it’s one thing to censor, another to report actual activity to law enforcement. I feel alternative methods should be used, and porn “to go” is a more realistic threat to minors than the Net is to a certain extend, yet the Net remains the king of content as always."



Google indeed issued a statement, sort of excusing the censorship under the statement of "the time has come to open ourselves to the Chinese market", and while their intentions make business sense, the outbreak had very positive consequences from my point of view - build more awareness and have the world's eyes on the Chinese enforcement of censorship practices, but is it just China to blame given "Western" countries do censor as well, or is it China's huge ambitions of maintaining a modern communism in the 21st century that seem to be the root of the problem?



In an article "A day in the life of a Chinese Internet Police Officer" I read some time ago, you can clearly see the motivation, but also come across the facts themselves : you cannot easily censor such a huge Internet population, instead, guidance instead of blocking, and self-regulation(that is limiting yourself with fear of prosecution) seem to be the current practice, besides jailing journalists! And while sometimes, you really need to come up with a creative topic worth writing about, free speech is among the most important human rights at the bottom line.



Chris Smith, Chairman of the House subcommittee that oversees Global Human Rights, proposed a discussion draft "The Global Online Freedom Act of 2006" "to promote freedom of expression on the internet [and] to protect United States businesses from coercion to participate in repression by authoritarian foreign governments". It is so "surprising" to find out that they are so interested in locating cyber-dissidents : "U.S. search engine providers must transparently share with the U.S. Office of Global Internet freedom details of terms or parameters submitted by Internet-restricting countries." exactly the same way I mentioned in my previous "Anonymity or Privacy on the Internet?" post.



Meanwhile, the OpenNetInitiative also released a bulletin analyzing Chinese non-commercial website registration regulation, giving even further details on the recent "you're being watched" culture that tries to cost-effectively deal with the issue of self-regulation :



"In a report published last year, “Internet Filtering in China: 2004-2005,” ONI shared its research findings that China’s filtering regime is the most extensive, technologically sophisticated, and broad-reaching Internet filtering system in the world. This new regulation does not rely on sophisticated filtering technology, but uses the threat of surveillance and legal sanction to pressure bloggers and website owners into self-censorship. While savvy website owners might thwart the registration requirement with relative ease, the regulation puts the vast majority of Chinese Internet users on notice that their online behaviour is being monitored and adds another layer of control to China’s already expansive and successful Internet filtering regime."



Yet another recent research I came across is a university study that finds out that "60% Oppose Search Engines Storing Search Behaviours", you can also consider the "alternatives" if you're interested :) A lots to happen for sure, but it is my opinion that personalized search is the worst privacy time bomb a leading search engine should not be responsible for, besides open-topic data retention policies and not communicating an event such as the DoJ's one, but complying with it right away, bad Yahoo!, bad MSN!



At the bottom line, Google's notifications of censored content(as of March, 2005 only, excluding the period before!), the general public's common sense on easily evaluating what's blocked and what isn't, and the powerful digital rights fighting organizations that simultaneously increased their efforts to gain the maximum out of the momentum seemed to have done a great job of building awareness on the problem. Still, having to live with the booming wanna be "free market" Chinese economy, and the country's steadily climbing position as a major economic partner, economic sanctions, quotas, or real-life scenarios would remain science fiction.



Technorati tags :
Privacy, Anonymity, Censorship, China, Search Engine Continue reading →

What search engines know, or may find out about us?

Today, CNET's staff did an outstanding job of finding out what major search companies retain about their users. AOL, Google, Microsoft and Yahoo! respond on very well researched questions!

Whatever you do, just don't sacrifice innovation and trust in the current services for misjudged requests at the first place from my point of view.

At the bottom line, differentiate your Private Searches Versus Personally Identifiable Searches, consider visiting Root.net, and control your Clickstream. You can also go through Eric Goldman's comments on the issue and his open letter regarding Search Engines and China.

As a matter of fact, I have just came across a very disturbing fact that I compare with initiatives to mine blogs for marketing research, EPIC has the details on its front page. It was about time a private entity comes up with the idea given the potential and usability of the idea. Could such a concept spot, or actually seek for cyber dissidents in restrictive regimes with the idea to actually reach them, besides mining for extremists' data? I really hope so!

Technorati tags:  

Privacy, search engine, Google, Yahoo, MSN, AOL

Continue reading →

Twisted Reality

I looked up the definition of Evil today, and I found it, I tried to play a Google War and came across 256 million occurrences of it, still there's a hope for all of us I guess. On the 17th of January I blogged on how China turned into the biggest black spot on the Internet's map, to find out that I even have activists commenting in my blog :)

Google has agreed to "remove certain sensitive information from our search results" you all know it by now, what you perhaps don't know is how what used to be the old Google still has its marks on the web. Google's Information for Webmasters still states that :

"Google views the comprehensiveness of our search results as an extremely important priority. We're committed to providing thorough and unbiased search results for our users."

I guess Chinese users should print this and stick it on their walls to remind them of the past as it says exactly the same. They have also removed their "censored notice" from "older removals", how come, and for what reason? Lack of accountability for when "local laws, regulations, or policies" were removing "sensitive information" before the date?! Google is my benchmark for disruption, but I guess its actions and "do no evil" motto were simply too pure for the business world, which on the majority of occasions is capable of destroying morale, even individuals..

Welcome in a "Twisted Reality" where one event looks like an entirely different one - on request, and the list is getting bigger!

But what is actually filtered in china these days, what are the topics of interest? Four years ago, a great initiative brough more insights into what's deemed "sensitive information", and while of course the list is changed on-the-fly, it is important to know how it blocks the top results, as this is where all the traffic goes.

Recently, CNET did a nice research on which sites are blocked by which search engine, I ever saw Neworder in there :)

The best thing about China's backbone is how centralized it really is and the way researchers are finding common censorship patters that could prove useful for future research. Is TOR with its potential applicable in China, and would initiatives such as the the Anonymous OS, or even TorPark, an USB extension of the idea, the future?

Meanwhile, in case they are interested parties reading this post, consider taking a look at the "Handbook for Bloggers and Cyber-Dissidents" courtesy of Reporters Without Borders.

Technorati tags :
privacy, censorship, search engine, google, china, TOR, Anonymity

Continue reading →

The Feds, Google, MSN's reaction, and how you got "bigbrothered"?

There's still a lot of buzz going on, concerning which search engine provided what type of data to law enforcement officials, and the echo effect of this event resulted in waves of angry end users, that among feeling "bigbrothered", now have yet another reason to switch back to Google, simple. MSN's silent reaction to this is the worst thing they could do given how actively they're trying to catch-up on search traffic. What did they provide anyway?

"Specifically, we produced a random sample of pages from our index and some aggregated query logs that listed queries and how often they occurred. Absolutely no personal data was involved. With this data you :

CAN see how frequently some query terms occurred
CANNOT look up an IP and see what they queried
CANNOT look for users who queried for both “TERM A” and “TERM B”

So picture, the following, "someone" requests his name, his friends' names, physical locations giving clues on possible area and while it isn't personal information(exact names, address etc.) it is personally identifiable one! If it happens once, it would become a habit, my point is that aggregating search info on ECHELON's wordlist is so realistic that you need a company to say NO, and evaluate the reactions of the others. The best thing is that I'm sure the majority of adult entertainment seekers don't need to take advantage of Echelon's Trigger Words Generator :)

Why you don't need to issue a subpoena to find out what's hot in the online porn world?

- take Google's advice into consideration, or start using Overture's keyword selector tool
- now ensure you have the most popular porn related keywords, and if in doubt, consult with an "insider" who would be definitely aware of what's hot, and who's to keep in mind
- use the first 20 pages from each popular search for your sample, these get the majority of traffic
- do a little research over Alexa to further back up your statements, and even use Google to measure the relative popularity of the first site that pop ups when you search for porn.
- ensure you have first consulted with traffic aggregators or paid reports on who's who online
- make sure before going online, another distribution vector so to say, the iPod is taken care of
- envision what's to come in the future, and mostly the interest and the social implications of these issues
- now, come up with ways to restrict children from using these going beyond the usual "But of course I'm over 21 years old" terms of use

What's to come up in the future? In one of my previous posts "Still worry about your search history and BigBrother?" I pointed out the possibilities for Search engines regulation and P3P, but the current self regulation is simply not working anymore.

Further resources on the topic can be found at :

Lorrie Cranor's Searching for Privacy : Design and Implementation of a P3P-Enabled Search Engine
PrivacyBird
An Analysis of P3P-Enabled Web Sites among Top-20 Search Results
Protecting Your Search Privacy: A Flowchart To Tracks You Leave Behind
Using search engines data, Google and forensics - clip

Technorati tags :
privacy,search engine,google,MSN,surveillance,porn

Image originally uploaded at Flickr by villoks

Continue reading →

Still worry about your search history and BigBrother?

The Patriot Search, recently started "helping" any government by making your search activity "public". Its search syntax terrorist:true *keyword*, and terrorist:false *keyword*, gives everyone the opportunity to be honest :) Why did the idea start at the first place? 

Because "only 4 out of 5 search engines allowed the government to see "private" user data". Though, a distinction between private searches VS personally identifiable searches should be made as well.

What's going to happen in the future? Search engines regulation, P3P, or stock market losses due to an initiative whose requirements I feel were totally wrong from the very beginning?

Consider going though David Berlind's comments as well!

Technorati tags :
google,bush,privacy,search engine

Continue reading →

Security quotes : a FSB (successor to the KGB) analyst on Google Earth

"Lt. Gen. Leonid Sazhin, an analyst for the Federal Security Service, the Russian security agency that succeeded the K.G.B., was quoted by Itar-Tass as saying: "Terrorists don't need to reconnoiter their target. Now an American company is working for them." A great quote, and I find it totally true. The point is, not to look for high-resolution imagery, but to harness the power of OSINT, improve their confidence by observing the targets "from the sky", and actually plan and coordinate its activities on huge territories. AJAX anyone? :)



However, the public has always been good at bringing the real issue to the rest of the world. There have been numerous attempts to spot sensitive locations, and I wouldn't be myself if I don't share the joys of the Eyeball Series with you. Of course, in case you haven't come across the initiative earlier.
However, the way it gives terrorists or enemies these opportunities, it also serves the general public by acting as an evidence for the existence of espionage sentiments, here and there. Echelon's Yakima Research Station was spotted on GoogleMaps, originally by Cryptome, see the dishes there? Any thoughts in here? Can Microsft's Local Live with its highly differentiated bird eye view on important locations turn into a bigger risk the the popularity of Google's services?



Technorati tags :

Google Earth,Google Maps,satellite imagery,OSINT,security,terrorism Continue reading →