Our yabbse Forum got shutdown!!!

[Ralph Button (rbutton)]

After running for 3 months our Cobraforum got shut down because it took too much cpu time to run it!!  We have 200+ active users on the forum ( a registered number of 2000+ ) and after 2 months 30,000 posts.  The ISP told us that they could no longer allow us to use the server, that we paid for time on, as the program was using up 80% of the processors times.  

It is with regret that I must inform you that it was necessary to disable your website today because it has crashed our servers both yesterday and today effecting 500 other paying clients.
 
This is not an issue of spamming or bandwidth abuse, but due to excessive CPU usage.
 
Your mysql database is within our 30mb limit and although your pumping over 40gb of traffic and creating a weekly log file of over 300mb we have not had issue with this until now.

We have documented this with the attached screen shots which coincides with yesterdays system crash and after turning off your domain our cpu usage went from 90.% down to a nominal 2%.

Bottom line, either it has bad or corrupt code, or is simply too inefficient to operate at it's current size.  
 

We WERE running 1.0 of YaBBSE.  Is there any hope that if I were to go to 1.3 it would be better?  Has anyone had a problem with an active forum of this size being shut down because of too much activity?  We didn't do many changes to the software.  Mostly ran it as is.  

[Jeff Lewis]

Again, we have about the same size forum and our server load is VERY low even with a few clients on it and in particular, one other site which has a huge SE board.

Send me over the screen shots  If you're stuck on a machine with 500 clients...

[Joseph Fung]

Hmm - that is interesting indeed.  Aer you sure you weren't  not running any other programs on your site?  Did you modify the source code in any way?

[Ralph Button (rbutton)]

The only "mod' to the code was to use the news function - with no fader - to place banner ads.  We were going to change that but just was waiting to load up a new PHP banner program.  We kept the template standard YaBBSE blue.  

Here is what they sent us:
The before picture

http://www.ctel.net/~rbutton/cobraforum%203-11-02before1.gif

The after shutdown picture
http://www.ctel.net/~rbutton/cobraforum%203-11-02after.gif

[Joseph Fung]

That's really strange.

What version of mysql were they running?  I ask, as it's a set of mysql processed that was creating the problems, so it seems to be the processing of some queries rather than an algorithm in PHP that did it.

Also, is there any way you could follow up with them to see if they've experienced the problem again - in case it was someone else's mysql processes?

Finally - was there any way to confirm that it was your site, rather than their mysql install causing the problems?  i.e. do they have other sites making extensive use of a mysql database? have they in the past?  is it simply possible that you were the first to really use their mysql install, and therefore you uncovered the fact that there was a glitch in their setup?

[Ralph Button (rbutton)]

No idea if others had been using the MySQL database but I am aware that they brought the forum down friday.  It cleared their problems.  They restarted the forum and in a few hours it brought the server down again.

Each time it came down there were 150 users on the system of them 80+ were registered.  Again I do not have access to the stats to look specifically yet.  They are going to let me bring it up online so I can pull the data off.  When I do I'll recheck the MySQL version but if I remember right it was the latest version available - again from my memory when I was setting up the forum.

Over the past month the system had been getting slower and slower.  Our users were very active and I noticed that the template loaded slower.  The top of the template loaded fast, then a longer wait for the active boards to load.  I reindexed it but no difference.  (Slower means about 30 seconds or more to get the screen up after logging in.)  Once in the system posting was taking about 10 seconds at most.  It was the screens fill in that were taking the most time to load.   And the fills in had nothing to do with the images - banner ads - as they were loaded up fast.  

That's the best information I have from memory as I can't access the site yet.

[Ralph Button (rbutton)]

The version of MySQL is 3.23.36

[Joseph Fung]

It sounds like the mysql queries were taking a long time to process, so it was taking a long time to generate the content tables for the page.  So the top would load fast, then it would crap out once it got to content...

Perhaps it was with the high user load, but I don't think so...I'll try running some tests.

[Ralph Button (rbutton)]

Joseph,
Thanks!!  I'm now contracting with the hosting site for a 1gig processor and 1gig of memory dedicated server.  
We're hurring to get the forum back on line.  Once it's on line you are welcome to look at the site and I'll give you admin rights to poke at it all you want.  I now have ftp access to the site.

Actual numbers, so I don't have to go by memory
21,789 posts in 2 months 1,450 registered members in 2 months, 11 boards, 5 categories, 91,806 log_topics are some of the details from the database.

The server stats page shows in 2 months 1,225,247 hits, 21,553 visits, 8,209,842 KBytes. We are avg 6003 hits per hour and 144092 hits per day.

[Ralph Button (rbutton)]

Well we're back on the internet.  New dedicated server, more memory, higher speed server, etc.  
So far everything is OK as I only have 50 users on the forum right now.

[Jeff Lewis]

Hehe keep us up to date and check on your server loads...

[Ralph Button (rbutton)]

It seems like when we hit about 100 users on the site the system starts to bog down.  We have a LOT of guests on the site that can look but not post until they register.  The users logged into the site are active and the post numbers reflect that.   About 300 posts since I got the system running on the new server Sunday afternoon.

I'm checking the database to be sure the indexes are OK.  Issue seems to be, with 100 users on the site, click on a topic and wait 20-30 seconds to get the posts.  Others tell me the 30second wait is more for them.  Dialup users have a MAJOR wait.

Should I upgrade to 1.1 or wait till 1.3 is available?  If you think 1.3 might help I'd love to test it for you!!  I have fully tested the backup of both the forum and database so can revocer easily from an upgrade oops!