Allow Google to index boards?

[I, Brian]

I'm under the impression that Google isn't indexing board content. I'm also under the impression that a specific hack would be required to allow it to do so.

As my sites are fairly small and new, it could help immensely to have my boards spidered for search terms.

I've seen that the server can be configured to allow this, but is way above my head. I've also seen reference on how changing the session info on the cookies can help.

If it's of any help in reference, php.bb's have a specific hack for allowing spidering:

#
#-----[ OPEN ]------------------------------------------
#  
includes/sessions.php

#
#-----[ FIND ]------------------------------------------
#
$SID = 'sid=' . $session_id;

#
#-----[ REPLACE WITH ]------------------------------------------
#
if ( $userdata['session_user_id'] != ANONYMOUS ){
  $SID = 'sid=' . $session_id;
} else {
  $SID = '';
}

Any thoughts?

[oldiesmann]

Try looking at http://www.google.com/webmasters/index.html

They have a lot of helpful info there, and an email address to contact them if you don't find what you're looking for.

[Spaceman-Spiff]

the code above only removes the session id from the url, which we don't show anyway

[David]

Google indexs many of the forums we host.

[I, Brian]

The actual board URL is indexed, but I'm after seeing the threads indexed as well. I've been checking and mine don't appear to be. Obviously, this is something I'd very much like to fix.

Any ideas on how to make YaBB SE more spider friendly for indexing actual threads?

[Surfy]

If you have a Domain for you Board, the Search Engines visit you often....

Surfy

[Darren]

How deep google crawls (and how frequently) all depends on the popularity of your site.

Google tends to do a deep crawl towards the end of the month, usually a few days before the dance.

It might just be that google hasn't yet deep-crawled your boards as yet......

I haven't yet had google crawl mine, yet the board index is indexed in google. I'd expect the rest to be crawled sometime soon.

[Daniel Hofverberg]

Although a few search engines won't index URL:s with question marks or ampersands, Google is not one of them - To my knowledge, Google correctly indexes URL:s with question marks, semi-colons and/or ampersands.

So YaBB SE forums should be indexed correctly in Google.

[I, Brian]

So YaBB SE forums should be indexed correctly in Google.

Has anyone checked, though?

The reason I'm asking is that I've been seriously looking into Google marketing the past few days.

Having a board main URL indexed is easy - but I'm strongly under the impression that YaBB boards are not being deep crawled unless they are specifically modified to allow it.

For example, I'm told there's a simple program for converting .php into static HTML pages. Without this trick (or similar functioning hacks, as with the php.bb board), then db driven boards simply will not be properly indexed - main URL, nothing else.

If someone can tell me I'm wrong then please do - but I'd rather see the issue proven, rather than assumed.

[Darren]

Once I've bee deep-crawled, I'll report back.

I know that my old YaBB Gold forum was partial crawled, which suprised me......and I'm confident that the PHP version will be fully crawled.

However, to be safe, you could apply a mod-rewrite on the URLs to remove all the ? and & etc

[I, Brian]

That's what I'm asking about - apparently, the whole indexing issue affects php.bb (with the hack) and vBulls as well (from what I've seen). In which case, what reason would there be for YaBB SE to be any different?

Oh, I know Google says it can handle ? statements - but there still seems to be potential issues with databases in general.

The original post asked if there's a work around.

I can't contribute much to the YaBB SE community - I can't code. But if this is a real issue this is perhaps going to be a helpful issue to address.

I've already copied my forum thread URLs straight from the board and into a HTML sitemap. The forum has been up for five months and appears un-indexed. I guarrantee it will be for the June Google dance.

[Surfy]

My Board is deepcrawled by Google, it works fine...

But i has one and a half year an tripod Account with the Board, in this time Google never has crawled my Board..

After i got a Domain Google cames without any Submission...

He crawl the Boardindex & the Messageindex & the Threadtopics of each public category..

Since then i have many 0Post Registrations  

Surfy

[Purple Raine]

To my knowledge, the boards do not get indexed (at least not individual threads).

Some of my threads are quite popular, and with a membership of 12,000+ we have enough links to and from our boards.  Although, I've seen individual boards being linked to, I haven't seen actual threads, and posts within threads linked to.

It would be awesome to have threads spidered.

[Surfy]

Threads yes, but not the Posts inside.. Only the Topics..

Surfy

[Spaceman-Spiff]

well, u cant expect google to store the information of all the pages on the internet
most likely, it will store the popular ones only...