Does anyone else notice the search feature being extremely slow the last 2 days...or not even working at all? I was trying to find some items on C5 stuff.

 

got over 100 new users the other day, that seems to be a trend now. traffic is going through the roof, and the truck site is going up too. Brains is doing all he can do for performance but there's only so much he can do with the vbulletin software.

 

Hmm, Corvetteforum.com doesn't seem to have any issues and they have 93,000 members and 11,000,000 posts. Something is holding back...but I can tell you this - it's not the software.

Seriously, this site makes how much from sposorship fees? For $700/month I could have a T1 to my house with a Server capable of handling a dozen of these sites with this demand. I'm sure I speak for quite a few members that are saying this is our favorite site, please stop limiting it to save a few dollars.

New posts feature disabled...

Search feature not working...

My 2 favorites and we can't use them.

I love this site, can I help by donating Server space?

 

Its not bandwidth, its server farming. I was talking to Brains about it the other day and currently Vbulliten apparently doesn't support multiple database servers running concurrently, hence why search is slowing down. With all the traffic there are only so many things we can do to maximize the performance while staying with VBulliten. To merge would not only cost money(less of the problem) but would also present a huge logistical problem as the database is enormous(2.2+ million posts).

I'll let Brains chime in and explain the nitty gritty of the whole deal, but thats the basics.

 

Very interesting. I know I sound like a pest but when you have Offtopic.com running vBulletin with 38,000,000 posts and 122,000 users on Database server that is a single P4 2.8E w/1MB cache, 2 Gigs of Ram, Intel CSA Gigabit Interface ,Quantum Atlas IV 10k 36k Raid 1 with Adaptec Ultra 160 Raid Card ...it just seems odd that 'tech doesn't run better.

I like talking this computer stuff...makes my day go faster.

 

Actually, LS1tech received:
July 4th....~35 new
July 5th....~48 new
July 6th....~70 new (so far)

Plus, the daily average Unique visitors to Ls1tech has hovered at 11,000 since May 18th, gone down to hovering at 10,000 on June 27th and has dropped below 10,000 since July 4th.

So, overall traffic has NOT gone up. The site does see weekly fluctations from night to daytime on average of 100 users online during the day to over 500+ on during the night.

Thanks for listening. I guess indirectly I am trying to help...and help others understand demand for the site. We love it!

 

Corvetteforum has search totally disabled for non-registered visitors. LS1Tech has search fully enabled for everyone, with full-text indexing. Also, Corvetteforum has tons of their threads in static archives (currently everything from before 2005, according to http://forums.corvetteforum.com/show....php?t=1125947 ); LS1Tech retains every one of its 3 million posts in the active database. And yet they still have plenty of search issues over there; thread today, with several responses which imply that search problems are a frequent occurrence there: http://forums.corvetteforum.com/show....php?t=1130256

Um, those specs are from the interview from April 2004 (in which it even mentions that new hardware was to be installed in July 2004). Here is some somewhat more recent information:
http://www.big-boards.com/servers.php
"Offtopic.com:
5 Web Servers each with Dual Xeon 3.0 Ghz Processors, 2GB Ram, Ultra 320 15k SCSI Drive.
Gallery/Uploads Server with Single Xeon 2.8, 2GB Ram, Ultra 320 15k Raid Array.
Database Server with Dual Xeon 3.0 1MB, 4GB Ram, Ultra 320 15k Raid Array."

Also, they frequently purge (not just archive) lots of threads there; and by its nature, search would be used far less extensively, and with less complex queries, at offtopic than at ls1tech.

http://www.big-boards.com/int.php?n=624&p=3
"Members are deleted every 30 days. So on the 31st day of inactivity they are deleted.
Posts in the main OT forum are deleted every 48-72 hours."

 

ACW, thank you for doing the grunt work on the research there. Was a little busy with my girls this afternoon to delve into this.

Let it suffice that we provide one of the most extensive and in-depth archived message board concerning LSx technology on the web that is NEVER purged, either posts OR members.

 

Thanks guys, saved me a bunch of typing

NOSJohn, you seem to know a little bit about a lot of things, but they don't relate to us -- I mean no offense, but your suggestions just won't work If you want some info on the limitations we're up against, check out vbulletin.com and vbulletin.org. Specifically, look for threads about those with "huge" sites (which they seem to think is anything over 1 million posts online). Look for some other admins begging and pleading with Jelsoft to port the software to use a better RDBMS. MySQL is great, to a point. We're getting past that point

I've worked both publically and privately with a few of the largest vBulletin forum administrators, and they all try to tell me the same thing "get rid of search" or "delete all your old posts." Tell me which of these solutions is good for LS1Tech?

We've been pushing the limits of the database engine for over a year and a half, and we're doing better than most if not all other sites with half as many online, fully searchable posts. There are two things we are currently up against -- the first is MySQL ISAM engine table locking, and the other is sheer disk subsystem data throughput. We have been known to completely saturate the SCSI bus with data from the 15k RPM drives pulling data at full tilt. I've been steadily increasing the query cache size to even attempt to keep oft-used queries in memory, which is basically the only thing that keep us afloat. Our key buffer hit rate is near 100% pretty much all the time (and over 400MB memory usage), and query cache hit rates are around 43% average (with just shy of 1000 queries in cache at any given time).

As an aside, yes we could easily afford a T1 to someone's house to host LS1Tech on. Problem is, we would require three T1's worth of bandwidth to even handle the AVERAGE traffic we see. In order to match the peak bandwidth pulled down during business hours, make that 7 T1's.

 

Ahh, you're looking at the numbers on big-boards Keep in mind, ever since the truck forums were moved that traffic no longer counts towards the LS1Tech totals.

Also, new user registration counts from the past 30 days:

 

Hey, no no no, I know much about nothing...give me some credit. In all actuality you could purge all the damn cam posts and cam questions and free up about 50% of your database space.

Archiving? Or take JRP's posts and all the other "stickies" that we know newbies don't read and create a sub-section similar to the "how-to" pages. Make them a page versus a searchable database item. One click to a page(s) versus several queries to the DB will lend a hand to decreasing BW. Perfect example - your Video page. Now, as opposed to people searching the Multimedia section, they can go directly to the video section and view.

Another example - your own diffusion of the truck forums - K.I.S.S., that's all I'm saying. The forum is getting so huge, it has to expand up and out...thereby creating sub-forums.

Big-Boards? What's that?

What are your thoughts on phpBB ?

 

archiving is an option, but forces people to search twice for things. Not something we really want to do. Dividing content on logical boundaries (ala the truck site) just makes good sense, and we will definitely do that where appropriate. We don't want to split so much there's no cohesion though. Oh, yeah. People don't read stickies, they use search A lot of times, there's things that just can't be answered in a sticky -- I use the search all the time myself. There's something you come across and you just want to see if someone else has touched on the same issue. Its not a fact of any kind per se, and for that you still need search. That type of search, and searches on specific users, makes up a VERY large portion of what folks are looking for. I know, I snoop the search all the time looking for ways to improve

phpBB? Ugh, talk about a low-feature security nightmare. I've bailed a few of those already, there's no way it would work for LS1Tech.

 

Ha - yes, let's ignore the stickies someone spent time and effort on to make it obvious and easier for the user. Ugh!

People love searching. Hence the growth of Google. Trivia - where did the term YAHOO come from? Can we bring back the "new posts" feature...please!

Low feature? Oh, by that you mean features that aren't already built-in...yes. I enjoy adding items and features to the phpBB, it allows me some creativity and sometimes a challenge. There were some security flaws in the earlier versions but as of recently, they are good to go.

Thanks for all the techie talk. It's good to speak to someone on the same level!

 

Interesting. Brian, does it help if we make all 2004 and newer threads read only?

 

Do you mean 2004 and older threads, read only?

 

If we make posts read-only, we have two methods of doing it. One which forces the user to search the archives separately -- this allows us the most performance, but with the biggest negative impact on the end user by effectively doubling the work involved with performing a search. The other is to make your old posts part of a completely separated table entirely, and update it as often as possible to keep the search data "fresh". I've already implemented the latter solution, which is how we kept search alive about a half million posts ago when we ran into big performance woes.

The BEST method is to have a new, dedicated server JUST for search. Its also the most expensive. I previously tried piggy-backing a slave database onto the primary web server, but it couldn't handle the load -- and performance was worse. It would need to be on a dedicated machine to be a viable solution.

 

How far back do the posts go? I know a few forums that only keep a year "on tap". This is due to all the same things being discussed over and over and over. Cams, wheels and tires, how big of a nitrous shot, what size turbo for my car, look at how fast my is, does my car look pretty, etc...there is much redundancy in forums ...to a point.

The good stuff - like JRP's compilations and others - can be kept. Make the new people READ !

 

AFAIK all the way back. Sometimes I get very few hits when searching, or just want to read a LOT on the subject and I find myself reading stuff from early 2002. That gets old quick, I don't know computers so I forget why....the board was switched over to vbulletin from something else so in the really old posts you get all kinds of funky lettering that I guess use to be smilies.

 

Yep, there's improperly-converted HTML code left in the posts from the UBB era. Almost everything back to the beginning in Nov 2001 is still here; the first few months of Racer's Lounge posts were removed (probably due to issues with the early UBB software), though.

 

To me that would seem like a trivial expense for how much money tech makes.

I vote for this solution.