Searchin'

Two ways to search the Web/Internet--
though they blur together

Indexes
- To the entire Web or one single site.
Subject Guides

Indexes
- Individual sites may catalog the pages on them
  - E.g., TCF389's search function
    - E.g., a search for "grades" turns up this year's and previous year's grade info.
    - You may use to add a search function to your final Web project with FreeFind. Beware: it does contain advertising.
- Or indexes may seek information throughout the Web and suck it into a huge database.
  - E.g., Google (my current favorite), Alta Vista, Hotbot, Infoseek, Lycos, Webcrawler, Northern Light
  - Remember: 2.8 million sites and 800 million pages
    - Of which search engines cover, at best, 16%.
  - Still, these indexes are enormous!
  - For more info on how they compare in size and feature, consult the Search Engines Features Chart
- How do they work?
  - A spider or Web crawler robot (a piece of software) connects to a site, follows links through it and collects text from the pages on that site.
    - Before crawling through a site, it checks for a text file called robots.txt--which can specify parts of the site that should not be indexed.
    - The Robots Exclusion Page explains how to control Web crawling robots.
  - Then that information is compiled into a large index which the user may search
  - Though the amount of information retrieved can be overwhelming
- Meta search engines
  - Aka, "All-in-One Search Pages"
    - E.g., Dogpile, Metacrawler, for a bunch more see the Yahoo! listing
  - You enter a word to search for in one place and it searches several indexes simultaneously.
    - And then collates the results.
- How can you, as Webmaster, can control the summary of your page that is listed in an index?
- Be sure to provide a <TITLE> tag
- Use <META> tags to control the site's description and keywords
  - HTML tags "hidden" in the <HEAD> tags that can contain additional information about your page.
  - Infoseek and others default to the first 200 characters from your Web page for the description, but you can change that with <META> tags.
    - <meta name="description" content="Write your description here">
    - Substitute your description for the italicized phrase.
  - Similarly, there's a <META> tag for keywords.
    - <meta name="keywords" content="Write your keywords here">
    - Substitute keywords (words associated with your site that users might search for) for the italicized portion.
    - Spam Penalty
      - All major search engines penalize sites that attempt to "spam" the engines in order to improve their position. One common technique is "stacking" or "stuffing" words on a page.
        
        E.g., "sex" repeated 100 times.
      - If the search engines spot a spamming technique, they may downgrade a page's ranking or exclude it from listings altogether.
  - Example from the MAR203 Concepts in New Media (U. of Arizona) homepage:
    1. <HTML>
    2. <HEAD>
    3. <META name="keywords" content="new media, Internet, Media Arts Department, University of Arizona">
    4. <META name="description" content="Syllabus and related materials for MAR 203 Concepts in New Media, Media Arts Dept., University of Arizona.">
    5. <TITLE>MAR203: Concepts in New Media</TITLE>
    6. </HEAD>
  - If you search Hotbot for "MAR203" it'll turn up listings based on these tags:
  - Creating META tags with Dreamweaver
    1. Go to the Insert menu and choose Head.
    2. Select Keywords and then type keywords for your site into the Insert Keywords box.
    3. Repeat step #1.
    4. Select Description and type a sentence or two that describes your site in the Insert Description box.
    5. You may add other META tags by choosing Meta after Insert and Head.
  - Meta Medic, an online service, will check your <META> tags for you.
  - There's also a <META> tag for excluding robots--though not many obey this:
Subject Catalogs or Guides
- E.g., Yahoo!, Excite, Looksmart, Infoseek (some do both)
- Information on sites is submitted or solicited and then someone categorizes it
- E.g., on Yahoo! you'll find the following category:
  - Regional > U.S. States > Arizona > Cities > Tucson > Education > College and University
  - The above goes further on down to
    - Public > University of Arizona > Courses
- Each word after an angle bracket is a category--becoming more and more specific
- Thus you may either browse through Yahoo!'s categories, or you may search it like you would search an index.
  - If Yahoo! does not find a category to match what you're looking for, it'll automatically search the Web for you.
  - And it'll give you the option of using other indexes and search engines
  - E.g., a search for "MAR203" yields
- How can you, as Webmaster, get your sites listed on subject guides?
  1. Submit them one-by-one to Yahoo! and the rest
    1. Try it: go to Yahoo's Personal Homepage's section and choose Suggest a Site (at the bottom of the page). Then fill in the form to register your own personal Homepage.
  2. Use a service that does several at once
    - Yahoo lists tons of them, but most charge for the service.
    - A few often limited services for free:
      - Submitside.com
      - Submitnow
      - Try it: Use one of these to register your site.

Search Logic

Regardless of whether you're looking at a Index or a Guide, the way your compose your search or query will greatly affect how successful you are.
The challenge is to find the specific information you want and not be overloaded with useless data.
Standard sentences do not work on most search engines
- A search engine is a piece of software that prompts you for words to look for (a query) and then goes hunting for you.
  - Nope: "I am looking for information about great danes."
- Instead, you enter words (aka, keywords) related to your search.
  - Yep: "dogs, great danes"
Most search engines allow you to combine those words by using certain Boolean operators
- E.g., AND, OR, NOT, NEAR
- Enabling you to narrow your search
  - Sink or Swim: Internet Search Tools & Techniques illustrates this quite well:

Boolean logic is the term used to describe certain logical operations that are used to combine search terms in many databases. The basic Boolean operators are represented by the words AND, OR and NOT. Variations on these operators, sometimes called proximity operators, that are supported by some search engines include ADJACENT, NEAR and FOLLOWED BY. Whether or not a search engine supports Boolean logic, and the way in which it implements it, is another important consideration when selecting a search tool. The following diagrams illustrate the basic Boolean operations.

AND

OR

NOT

Search Engine Watch, a site devoted to all manner of cool search advice, explains the specifics of how particular search engines implement Boolean logic:
- Power Searching for Anyone

Sample Searches

Search for MAR203's home page, which has been up since early July 1997--using the search term "MAR203". Here's how the various engines display the main MAR203 page. Note: Google's cache and AltaVista's translation service.

Google

MAR203-01 SS II 1999: Concepts in New Media
Syllabus and other resources for a course on computer-mediated communication in the Media Arts Department, the University of Arizona.
www.arts.arizona.edu/mar203/sum199901/ - Show matches (Cache) - 7k - Similar pages
[ More results from www.arts.arizona.edu ]

Alta Vista

MAR203: Index to MAR203
Syllabi Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona....
URL: arts.music.arizona.edu/mar203/index.html
Last modified on: 4-Mar-2000 - 3K bytes - in English
[ Translate ] [ More pages from this site ] [ Facts about: University Of Ari... ]

Excite

MAR203: Concepts in New Media - Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona.
http://arts.music.arizona.edu/mar203/Sum199801/index.html

Infoseek

MAR203: Concepts in New Media

Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona.
76% http://www.arts.arizona.edu/mar203/ (Size 4.5K)

HotBot

1. MAR203: Concepts in New Media

99% Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona.
http://www.arts.arizona.edu/mar203/, 4602 bytes, 21Jul97

MetaCrawler

Query: mar203
Collated Results: 1 to 6 of 6 references.

MAR203: Concepts in New Media
AltaVista: Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona.

Infoseek: Syllabus and related materials for MAR 203 Concepts in New Media,Media Arts Dept., University of Arizona.

1000, http://www.arts.arizona.edu/mar203/ (AltaVista Infoseek)

Try it: Search Exercises

Want to look over a stranger's shoulder while he/she searches?
- MetaSpy and MetaSpy Exposed!
- Web Watcher's Search Voyeur
- Oodles of voyeuristic fun.
Scavenger Hunt: Web Scavenger Hunt on WebCT

Using the Web to search for non-Web info

Looking for people's phone numbers, addresses, or e-mail accounts
Try ego-surfing on: