 |
|---|
Published @ searchlores in January 2003
|
|---|
|
First of all, let's clarify something : we are not working "FOR" anybody, we are not doing anything for money, hence we are not "professionals", we are just harmless (dangerous) fools :) (aha... only in part serious...) As for 'me', i am one of 'we', i'm a fool who has a hobby for fields related to intelligence, and for playing with (key)words. About the stuff listed there, i've not (yet) read even 0.1% of it, but i think that doesn't mean i can't process and evaluate the information i am able to find. Anyway i sincerely hope the documents you'll find using the tecniques explained in this essay will be of some use for students, researchers, game masters in need of inspiration, whatever... The first time i've hit a military field manual, it was the Psychological Operations manual, which has the reference FM 33-1-1 (you'll be able to find it on the web, here a local copy just to give you an exact idea of what we are discussing here). Such an easy to figure classification made me thought of a way to fish the whole collection, and moreover, as the presence of the psyop manual in the open source web (the indexed, not hidden and not unlinked part of the web) shows, some of them are said to be of 'restricted access'. Check this official .mil site for confirmation. Try to download some part of the manual.. Here are some of the most useful site i've bookmarked during my searches, and i know they have less value than some gems that still await to be found in this funny field :) http://www.globalsecurity.org/military/library/policy/army/fm/ I think there are more than 2 Go of field manuals and other military documentations in the urls given above (including the FM 33-1-1 we saw above). If you do take a close look there, you'll find again and again some restricted documents, not yet unclassified. So what? Do you really believe 'they', also the services "specialized" in information management, ignore that? Are they klebing us on the web? It's a kind of huge trolling operation? Am i being too paranoïd? The juicy stuff should be on SIPRNET in my humble opinion. And if they aren't aware of this lost of secret informations, i eat my mouse, with the wire. But anyway those documents, even if "restricted", are now quite unrestricted, opened like dead shells, thanks to all s.e. crawlers and spiders :) In order to read them you don't even have to visit (and surely you do not need to break inside) a .mil site! You don't even have to use proxies, nor proxy chaining bazaars! Just use google's cache and you will consult the manuals remaining fully anonymous for the .mil sites (and if you do not trust google cache either, just find one of the 1000 copies of your target that have already been stored on those pesky private 'homepages' that blossom all over the web :-) This said, i noticed that the number of "restricted documents" in open source information are decreasing over time inside the indexes of the search engines (creeping censorship?). So, if you have a good connection, and a good list of proxies, please, by all mean, download everything you can... and mirror everything on some pages, wich you'll later get indexed by the search engines (and should google, for some reason, "disdain" such indexing, he'll get for sure some "real" competition by the older search engines: commercial search engines' bickering helps freedom of expression! :-) 
Let me give you my 'stratagems' to fish such documents quickly. I believe more useful to KNOW that these documents exist, and that one is able to fish them with a simple search engine, than just to distribute such files in a 10 Go package. Distribution is always stale per definition. Moreover such "knowledge approaches" do not have the same legal consequences :) One of the first queries i made was this one : +filetype:pdf +"DISTRIBUTION RESTRICTION" -unlimited. Here it is. Note that the filetype:pdf option, that asks to give only results of pdf type, only works with some search engines (for instance google). These engines can also search INSIDE powerpoint slides (filetype:ppt) word documents (filetype:doc), excell sheets (filetype:xls) and also poscript file (filetype:ps). A nice feature, if you realize that "military users" (as well as bureaucrats and zombies) simpky love those pesky microsoft and adobe formats (i'm not talking of poscript). Test "filetype:ppt site:.mil" on google : 41.600 (my time, my home). Moreover, as i said earlier, google offers the opportunity to view the cache of the target pages, and moreover converts pdf files into html. Nothing to install, not many traces left behind. "DISTRIBUTION RESTRICTION" are common keywords on military documents, generally followed by the type of restriction that applies to the current document. Therefore, filtering with -unlimited should reduce the results to (only) "limited distribution stuff". (I am sure ~S~ Jeff will soon experiment on this) What about the field manual FM-46-2 Unlimited control of the world. We are the borg, you will be assimilated? ;) Dig in the results, there are things like "Distribution Restriction : NONE". And now, how're yu supposed to filter that? -none is totally unthinkable for obvious reasons. Finally 290 hits, for this 'advanced' (using the advanced feature of google) query.. should even only a meager 10% of the results point to "restricted documents", that means a lotta files nevertheless, and hence a lotta sites that 'potentially' offer more results of that type of data for those that have learned the simplest 'ways' of searching :-) One can build variations on such a query, in order to generate a more consequent information environment. For instance: (filetype:pdf OR filetype:doc OR filetype:ppt) AND "DISTRIBUTION RESTRICTION' -unlimited Even simplier : +"DISTRIBUTION RESTRICTION: Distribution is authorized to" Observing the 'template' of the field manuals, and especially those used in those pesky "restricted distribution documents", one can exctract a group of (key)words that will litterally CUT through the web pudding. This is a quite intuitive method, pointed out by ~S~ fravia+ in his 'synecdochical' approach. Let's take as our model the (Mine/Countermine Operations) FM, that you, by now, should be able to find on your own. Suppose that this portion/box/snippet appears ALSO on the majority of all "Field Manuals" :
----------------------------------------------------------------
DISTRIBUTION RESTRICTION
Distribution authorized to US Government agencies only to protect
technical or operational information from automatic dissemination
under the International Exchange program or by other means. This
determination was made on 1 July 1992. Other requests for this
document will be referred to Commandant, US Army Engineer School,
ATTN: ATSE-TDM-P, Fort Leonard Wood, Missouri 65473-6650.
[ 8< ]
DESTRUCTION NOTICE
Destroy by any method that will prevent disclosure of contents
or reconstruction of the document
-----------------------------------------------------------------
Wich is quite different from "DISTRIBUTION RESTRICTION: Approved for public release; distribution is unlimited." "DISTRIBUTION RESTRICTION: Distribution is authorized to" should result into a great filter in order to remove web noise. The whole pack inside one single sentence. The fished documents will probably be military documents, not approved for public release, since their "distribution is authorized to" (only) someone/somegroup.. Quite a simple query, and a powerful one as well (we may imagine many other fields where we would be able to apply a similar searching trick :) Even more powerfull, and funnier : Search for documents saying that they should be destroyed, by any method. fnord! Isn't that a bit magical ? :) "Destroy by any method that will prevent disclosure of contents or reconstruction of the document" : 252 hits on google... not even FM centered this time :) 
Most of the information isn't really hidden. It's there, not so far from us, but under a ton of noisy, unrelated, data. The major difficulty you face while hunting data is not of technical type, but is more cognitive. You are limited by your IMAGINATION. I hope you'll find these stratagems (or rather strata - gems) somewhat interesting, and we hope you will contribute by sending as feedback OTHER good urls, queries, or documentation (with a description of the way you used to find them). Ah. To conlude. It's like the Ocean, hu: Sometime, the web can be a little bit scaring, and sometime really beautiful. It's like the Mountains, hu :) During the writing of this article, i've found something enlightening : "a wild flower surrounded by metal fences". I would like to offer it to you. Perform the query seen earlier, and check the result #19, aha. :) ~S~ loki / aRT |