My Website and search engines - advice please?

Discussion in 'other security issues & news' started by HD rider UK, Mar 10, 2005.

Thread Status:
Not open for further replies.
  1. HD rider UK

    HD rider UK Registered Member

    Joined:
    Feb 16, 2005
    Posts:
    121
    Location:
    Gloucestershire, UK
    Hi all
    Im not sure if this is the correct place for this question, so input from the mods is welcome.
    In brief, i have a small website on custom biking (url in my profile)and recently have been getting emails from what appears to be a search engine regarding updating my listing on their service. Couple of things make me a bit suspicious here:
    The thing is that I have never submitted the site to them (or anyone else for that matter)
    the email headers have the return addy as
    <owner-LISTINGS@WWW--SEARCH.COM> strange format?
    and also sometimes
    www--search.com (ns1.www-goto.com [202.14.69.2])
    Now i have googled these and it doesnt seem to clear as to what/ who they are, there is some link with overture i think, and some suggestion that they (goto) might be a bit iffy.
    Any ideas?

    jock
     
  2. Spambots.

    They crawl websites, grab emails of the webmasters, then send spam.

    PS You don't need to submit your sites to search engines, most search engines have automatic web crawlers that index pages as long as you are linked from a site that has already being indexed. This includes google.
     
  3. HD rider UK

    HD rider UK Registered Member

    Joined:
    Feb 16, 2005
    Posts:
    121
    Location:
    Gloucestershire, UK
    Thanks for that, I was a bit puzzled as to what was going on.

    Jock
     
  4. meneer

    meneer Registered Member

    Joined:
    Nov 27, 2002
    Posts:
    1,132
    Location:
    The Netherlands
    Use a robot.txt file in your web root directory to limit access to your site for robots and spiders: Tutorial
     
  5. HD rider UK

    HD rider UK Registered Member

    Joined:
    Feb 16, 2005
    Posts:
    121
    Location:
    Gloucestershire, UK
    thanks meneer, will do

    jock
     
Loading...
Thread Status:
Not open for further replies.