[ AltaVista] [ Advanced Query]  [ Simple Query] [ Private eXtension Products] [ Help with Query]
AltaVista: Submit New URL

Submitting New URLs

When you submit a URL, we attempt to fetch the document immediately.

We also accept submissions of URLs through Scott Banister's Submit It! service.

Excluding Pages from the Index

Our spider will find any URL connected to the main body of the Web through even one link. If you don't want your entire site to be indexed, we strongly advise that you take advantage of the Robots Exclusion Standard by setting up a /robots.txt file. It only takes a minute, and gives you complete control over what fraction of your site is indexed. The file looks like:
User-agent: *		# directed to all spiders, not just Scooter
Disallow: /cgi-bin/sources
Disallow: /access_stats
Disallow: /cafeteria/lunch_menus/
Any URL matching one of these patterns will be ignored by robots visiting your site. This file is read by Scooter every few days, so changes may not take immediate effect.

Surprise . Legal . FAQ . Add URL . Feedback . Text-Only
DIGITAL Copyright © 1996 Digital Equipment Corporation. All rights reserved.