If there is one question that is often asked in the squidu forums, it’s this one: How can I get my lens indexed in google? Or, why hasn’t google indexed my lens yet?
I always tell people the same thing: get links to your lens. Start blogging. At the same time I know that with my online profile indexing is rarely an issue. When it comes to indexing new content on old lenses I also don’t complain much. Perhaps because I just take into account that search engines are slow. Perhaps because I generally make sure that my new content gets links.
Anyhow, I recently topped 300 lenses and decided to get myself a second squidoo account. And just to show how slow search engines can be, I’m publishing this post only when it’s indexed by google. Note that the links to the below pages will help this new account, but do not help it get indexed as that has already happened by the time this is published.
Date line:
Dec. 18th – start of new account
By chance there’s a google cache of my lensography of lensographies on that date and I see I did not link to this account on the 18th itself, though I did link on it somewhere that day or the next.
I also wrote about my new account on squidu that same day.
I started out by transferring my three advent calendar lenses to the new account. I wanted to go slow, because in the Christmas season I did not want to screw up the rankings of my lenses in google. So I went with low traffic, low sales lenses first. Understandably going slow may also slow the speed of Google and the other search engines.
Dec. 25th
Yes, I actually checked this on Christmas day! There’s no family over just yet. I expect them by lunch. Anyhow, I have now transferred 8 lenses to my new account, but Google has only cached one of these lenses as having the new lensmaster picture and profile. No accident that this is the highest ranking lens and the one that had massive traffic just before and on the first of Dec. In other words: google checks high ranking lenses for updates more often than other lenses. Even if their link profile is very comparable.
The new lensmaster profile is not yet indexed in google. The squidu forum thread where I wrote about this new account IS indexed. So google knows about at least two links to my new account, but hasn’t seen fit to index it yet.
Searching the URL of the new account in Yahoo DOES give a result. Apparently yahoo was faster in this case in spidering new content than Google was. The cached result in yahoo is not dated, so there’s no telling when yahoo last spidered it. Still, it was faster than Google.
Note that yahoo may be fast, but it does not recognize the links to the lensmaster lenses. It only sees the links we enter in our profile, not the list of all lenses. This is because those links are in javascript.
Bing (formerly MSN Live Search), like Google, has not yet indexed the new account. Also, like Google, it DOES recognize javascript enough for the links to all my lenses on my spirituality account to show up. And like Google it has a date on the cache. Last indexed: 24 dec. That’s better than both Yahoo and Google. (Judging yahoo by the amount of lenses it shows for my spirituality account: it’s off).
Dec. 31st
I’ve just checked again, and now my new account IS indexed. However, looking at the date it appears that it was indexed precisely on the day one would expect: 24th of December when the one lens on the account that had been recently indexed was indexed. HOWEVER, that did not mean it was showing in the Google index publicly.
In other words: I’m forced to the conclusion that Google indexes things without always showing those things in the index that searchers get to see. For a more detailed analysis I should have checked the various Google datacentres, but that’s beyond the scope of this article. For every day SEO it’s enough to note that just because Google doesn’t show a page, that doesn’t mean it hasn’t yet indexed it. It merely means that, and this is disturbing enough, it doesn’t think the page important enough to be shown to searchers.
This brings me back to the basic advice I always give people complaining about not being indexed: make sure Google thinks your page is important, by getting links to that page. Note that on the 26th I blogged about my new lenses and my new account. That link was not necessary, in hindsight, to get the new account indexed. However, it may have been enough for Google to show the indexed page to users.
All in all it took Google 6 days to index my new account and somewhere between 8 and 12 days for it to be shown to users.
Notes:
1) How to check whether a page is indexed and shown to users. Search site:http://www.squidoo.com/whateveritis
The result, for lenses, should include only your lens and will usually also include the alternative URLs for that lens that the ‘Explore related pages’ feature produces for your lens.
2) How to see when your page was indexed last:
To see when your page was indexed last, click ‘cached’. On the cached page you will see the date it was cached at the top and the content of the page as Google knows it below.
{ 5 comments… read them below or add one }
This is exactly what I look for when I seek SEO information; research, testing and results rather than rumors, wild theories or doomsday warnings without any evidence to back them up.
Congratulations on a worthwhile and well document project. It will help a lot of us.
K,
In the Google results, the line which reads, “www.flickr.com/ – 10 k – Cached…” at the end there is often a little button with an arrow pointing upward. I’ve seen a discussion that you can simply vote a page up. Do you know what I’m talking about or what the little arrow means?
Brenda
That little arrow shows you’re logged in. It affects ONLY your personalized results. Personalized results were upped for general users in 2009 and are definitely something to take into account for the future too. However, there is little indication that we can affect what other people see in THEIR personalized results, so for now, aside from keyword research, I ignore it.
If you want to make sure your results are NOT personalised and NOT regionalised, check scroogle.
Your info is very useful, I also find the same things, yahoo spider is certainly faster than google, my lense on “wind power cost”, also the keyword is, has already rank on first page in yahoo, but in google very low.
Hi Aachen712,
This was not a representative sample. It was just an experiment. I don’t think that Yahoo is generally faster in indexing. In fact, my example shows that Google knew about the page almost as soon as it was up and cached it. It just didn’t show it to searchers yet.