SEO 2020 FORUM Forum Index SEO 2020 FORUM
SEO Training, SEO Tips, SEO Strategy
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

 

SEO 20/20 Books & Reports

 

How do I know when my site has been spidered?
Goto page 1, 2  Next
 
Post new topic   Reply to topic    SEO 2020 FORUM Forum Index -> SEO Resources & Advice
Author Message
OnlyTopResults
SEO Yellow Belt


Joined: 07 Jul 2006
Posts: 28

PostPosted: Thu Aug 10, 2006 3:27 am    Post subject: How do I know when my site has been spidered? Reply with quote

OK - let's say I have uploaded my site and taken a couple of weeks to submit it to free Directories. I can use http://www.marketleap.com/publinkpop/ to see when back links have started to appear, though this of course doesn't give me any indication when the SE bots will crawl those Directories next. Presumably if I only submit to PR1+ Directories it will only take a matter of a few days.

So, I have access to an analog stats package on my site hosting account and can see loads of info there! But what am I looking for? Presumably the advice is to let Google, MSN and Yahoo spider the site and then start the next phase of adding a menu, adding an xml sitemap, posting to Article Directories, etc.

How do I know when those 3 key 'bots have been through my site?

Thanks
Adrian
Back to top
Charles Heflin
Administrator


Joined: 16 May 2006
Posts: 337

PostPosted: Thu Aug 10, 2006 8:34 am    Post subject: Re: How do I know when my site has been spidered? Reply with quote

OnlyTopResults wrote:


How do I know when those 3 key 'bots have been through my site?

Thanks
Adrian


You will see yourself indexed in each of the search engines. Go to Google and do a quesry on your site's url. If it appears then you are ready to go. Same with Yahoo and MSN.
Back to top
OnlyTopResults
SEO Yellow Belt


Joined: 07 Jul 2006
Posts: 28

PostPosted: Thu Aug 10, 2006 9:13 am    Post subject: Dooooooohhhh!!!! Reply with quote

Do you know, it never occurred to me that the answer would be SO simple! Confused

Of course Googlebot seems to take twice as long as the other 2 and so I'm having to be patient!

I've just added a spyware-free very useful toolbar from http://www.rankquest.com/download-toolbar.html which also allows me to easily check the status of my sites with these top 3 SEs. This is only one really cool tool in the toolbox though - see http://www.rankquest.com/seo-tools.html Highly recommended!

Cheers
Adrian
Back to top
BWWD2006
SEO Yellow Belt


Joined: 14 Aug 2006
Posts: 34
Location: Algonac, Michigan USA

PostPosted: Mon Aug 14, 2006 11:39 pm    Post subject: Reply with quote

If you want to check when your site gets spidered by different search engines consider checking out Bot Monitor, this is a PHP script that is included in your php pages and as soon as a search engine spider pays a visit to that particular page you receive an email informing you of what search engine spidered your site and what page it spidered

This can be especially useful in determining which pages search engines are seeing and which engines are visiting, it even informs you when some blog and feed spiders visit also, check it out

***UPDATED***
check http://www.seo2020.com/seo-forum/viewtopic.php?t=290 post for a discount code to use when purchasing any of my products and services to get 25% Off


Last edited by BWWD2006 on Wed Aug 16, 2006 7:11 pm; edited 1 time in total
Back to top
Ahri22
SEO Blue Belt


Joined: 01 Jun 2006
Posts: 163
Location: Australia

PostPosted: Tue Aug 15, 2006 5:18 am    Post subject: Reply with quote

Don't be afraid to submit to PR0 directories, either! Especially if they're reasonably new (you may want to see whether they're indexed in Google though...). A site can go from PR0 to PR4 quite quickly, and if you get your sites listed while the directory's still new, you'll most likely get your sites on the main pages (more likely to get PR when updated) and you'll also be more likely to get accepted by the directory quickly. The backlinks won't count for much now, but they could be valuable later!

Cheers
Fiona
Back to top
OnlyTopResults
SEO Yellow Belt


Joined: 07 Jul 2006
Posts: 28

PostPosted: Tue Aug 15, 2006 5:34 am    Post subject: thanks for that Reply with quote

BWWD2006 wrote:
If you want to check when your site gets spidered by different search engines consider checking out Bot Monitor, this is a PHP script that is included in your php pages and as soon as a search engine spider pays a visit to that particular page you receive an email informing you of what search engine spidered your site and what page it spidered

This can be especially useful in determining which pages search engines are seeing and which engines are visiting, it even informs you when some blog and feed spiders visit also, check it out

Looks like a very useful tool. However at $5 a site I can see it building to be a bit of a cost over time. I will scout about and see if I can find something similar, meantime the tools at http://www.rankquest.com/seo-tools.html will allow me to check, although they are not as proactive as the system you mentioned.

Cheers
Adrian
Back to top
OnlyTopResults
SEO Yellow Belt


Joined: 07 Jul 2006
Posts: 28

PostPosted: Tue Aug 15, 2006 5:36 am    Post subject: A useful thought Reply with quote

Ahri22 wrote:
Don't be afraid to submit to PR0 directories, either! Especially if they're reasonably new (you may want to see whether they're indexed in Google though...). A site can go from PR0 to PR4 quite quickly, and if you get your sites listed while the directory's still new, you'll most likely get your sites on the main pages (more likely to get PR when updated) and you'll also be more likely to get accepted by the directory quickly. The backlinks won't count for much now, but they could be valuable later!

Cheers
Fiona

Yep - I had missed the point that a PR0 need not be an automatic eliminator as long as a DIrectory has been indexed. Good point Fiona!

XXX
Adrian
Back to top
BWWD2006
SEO Yellow Belt


Joined: 14 Aug 2006
Posts: 34
Location: Algonac, Michigan USA

PostPosted: Tue Aug 15, 2006 9:04 am    Post subject: re bot monitor Reply with quote

Mistake on page should have read 10 site licence not 1 lol

have corrected
Back to top
OnlyTopResults
SEO Yellow Belt


Joined: 07 Jul 2006
Posts: 28

PostPosted: Tue Aug 15, 2006 9:16 am    Post subject: That's a deal! Reply with quote

That's more like it Michael! I just went to purchase my 1st licence, but I am unable to tell you what the URLs might be for 10 sites right now - I have 2 in the works but beyond that I have no idea!

Your purchase page says "Additional Info (e.g. Url For Site Licence, Submission etc)" ... how do I get around this?

Cheers
Adrian
Back to top
BWWD2006
SEO Yellow Belt


Joined: 14 Aug 2006
Posts: 34
Location: Algonac, Michigan USA

PostPosted: Tue Aug 15, 2006 9:19 am    Post subject: re Reply with quote

ahh lol just buy it and work that out later, just drop me a note when you use a new site

hope you enjoy the program

check this link below for discount code to get 25% Off any of my products and services

http://www.seo2020.com/seo-forum/viewtopic.php?t=290
Back to top
Display posts from previous:   
Post new topic   Reply to topic    SEO 2020 FORUM Forum Index -> SEO Resources & Advice All times are GMT - 4 Hours
Goto page 1, 2  Next
Page 1 of 2

 


Powered by phpBB © 2001, 2005 phpBB Group