Alfa Romeo Forum banner

1 - 20 of 23 Posts
P

·
Guest
Joined
·
0 Posts
Discussion Starter #1 (Edited)
I'm not trying to be a smart**** but am putting forward what MIGHT be causing the problems with AO lately. Data corruption can be a result of what I will try and explain. I will try and make it as least dry and mumbo jumbo as possible. Please any IT people do chip in.

The size of threads should have no impact UNLESS the database has not been normalised, (designed), properly in the first place or corruption of data has taken place.

Redundancy of data is the biggest culprit in the corruption of data. Everybody on the AO database has a unique ID number. Let's say 12345. There should be only ONE main record with that number on the whole of the database. A well designed database means each record is is linked to the ID number field on the main record rather than repeating the data. This is VERY important. For example the main record will have your avatar on it.

For example, a really poorly designed data file would have a main record with 12345 as your ID. number with your Name and Address details for example. In the poorly designed file each one of your threads would have a field with 12345 in it plus each one of your posts would have a record with 12345 in it. Say you have 500 threads and you've made 1000 posts. The poorly designed record would have 1 main record for you plus 500 for each thread plus 1000 for each post. All told that means your avatar is repeated 1501 times PHYSICALLY on the database while a well designed one would only have 1 PHYSICAL record with 12345 on it and your avatar and the rest of your threads and posts would be linked to that one record. Linking rather than separate redundant data is essential. The above is a silly example to try and get the point across.

Redundant data leads to corrupt files and corrupt files lead to slowness and what has been seen lately. I think further posts in this thread are more clear.
 

·
Banned
Joined
·
57,288 Posts
So good it had to be posted in two threads :)

Why bother searching the multitude of why AO is slow threads, I'll just make my own :thumbs:
 
P

·
Guest
Joined
·
0 Posts
Discussion Starter #3
So good it had to be posted in two threads :)

Why bother searching the multitude of why AO is slow threads, I'll just make my own :thumbs:
After it was pointed out that locking the long threads made some difference, corruption has certainly come up as a prime reason.

Orphan files as they are described, really slow a system down. These are records that don't have a main record to link with.

I'm not making this up. I was involved in the management and design of a $200m banking system in the past even though I'm not in IT anymore.
 

·
Banned
Joined
·
57,288 Posts
After it was pointed out that locking the long threads made some difference, corruption has certainly come up as a prime reason.

Orphan files as they are described, really slow a system down. These are records that don't have a main record to link with.

I'm not making this up. I was involved in the management and design of a $200m banking system in the past even though I'm not in IT anymore.
And I built a RR supermarine spitfire with a knife and fork :p
 

·
Registered
Joined
·
11,604 Posts
Why only at certain times as has been recorded? :confused:
 
P

·
Guest
Joined
·
0 Posts
Discussion Starter #6 (Edited)
Why only at certain times as has been recorded? :confused:
It happens at peak times I believe. Any software will handle a certain amount of data corruption until it becomes bamboozled and then thrashes about.

I know we've had our run-ins and I'm truly serious about this and hope you listen. I will use you as an example.

When you have registered on AO there will be a 'parent' record set up giving you a totally unique number. For your example let's say 54321. On this parent record will be your name, avatar, email and basically parts of your profile that are unique and usually static.

Each time you generate a thread and/or a post obviously the 'child' record which is the thread or post does not want to duplicate your avatar and status, signature etc. That is why each child record will have a field with 54321 on it and this will link to the SINGLE parent record. This is why when you change your avatar/signature/status etc. there is only ONE physical avatar held on the database but if you change that avatar and you've made 20000 posts all the posts will immediately reflect the new avatar. It is a parent/child relationship. A parent record can have many children but a child record can only have one parent.

Problems arise when the data is corrupted. This can be an orphan(child) record without a main parent record. It can also happen if two or more parent records have the same number i.e. 54321. Duplication of identification data is real trouble.

It would take a whole bunch of corrupt data to start affecting the speed and access to the database. It could be that recently a lot of records were lost or if the main record file is corrupt there could be a whole pile of parent records with the same identification. Let's say for example 500 records with an ID of 54321.

The time element MAY be due to the fact that the application has been designed to use a particular partition of data to ease the burden in peak hours. If that partition is corrupted the fun and games start.

To summarise... each AO member has ONE parent record which can have thousands of child records. This parent record should be unique and the uniqueness is the ACCOUNT ID. (e.g. 54321 above.) Corruption occurs when you have child records that have no link to a parent or parent records that are not unique. i.e. more than one with an ACCOUNT ID. of 54321 for example in above example.

I hope this is making sense to people. Obviously the application is much more complicated than that but the general theory is still the same in every database on this planet. A bank for example gives you a unique account number which is the parent record and every time you make a transaction a child record is linked to the parent record. Imagine if two people would accidentally have the same account number. The software has no way of knowing which one is which and will produce error logs that depending on how many will slow down everything. The same goes if the banking software finds a whole heap of transactions that aren't related to any parent on the database. It has no clue who made those transactions.
 

·
Registered
Joined
·
11,604 Posts
Am listening - I ain't always non PCPT ;)

Peak times I would say are 'random' (ignoring general weekend / mid BST evening) compared to the 'specific' times posted by folk, specifically twenty odd minutes past certain hours...

Speculation aside, surely there are error logs that are now boringly repetitive?

I would like a definitive answer TBH, not unreasonable considering the hassle using the site for weeks...

Fair play to BobdaMod for the severance of the mahoosive threads, tried summat and so far, site is way better :thumbs:
 
P

·
Guest
Joined
·
0 Posts
Discussion Starter #8 (Edited)
Am listening - I ain't always non PCPT ;)

Peak times I would say are 'random' (ignoring general weekend / mid BST evening) compared to the 'specific' times posted by folk, specifically twenty odd minutes past certain hours...

Speculation aside, surely there are error logs that are now boringly repetitive?

I would like a definitive answer TBH, not unreasonable considering the hassle using the site for weeks...

Fair play to BobdaMod for the severance of the mahoosive threads, tried summat and so far, site is way better :thumbs:
There may be error logs of different complexities depending on how the application was designed. The time factor is probably automatically determined by the software.

To be quite honest if chopping off long threads makes a difference and there is no data corruption then the system hardware is not up to it. If the hardware is up to it and the problem is fixed by chopping down the threads these are probably only band-aid solutions and this is no criticism of Bobda trying different things.

The AO database is basically small compared to zillions of others including many other forum databases.

As for definitive answers only by close perusal of the application software can that be done. It may be the finest software in the world but if data corruption has occurred, which sounds the case to me, it will only just keep on chugging along.

To really determine that the data hasn't been corrupted a diagnostic program needs to run through the entire database on either a backup or when the system is offline to look for problems in the relationships of all records or crap data in certain fields. Believe me when an identification field expects a unique number from 0000000 to 9999999 and somehow @#78_89 has got in there it causes the program to pause or blow up. If a diagnostic program is not available one should be written as quickly as possible. It will be invaluable in the future. Shouldn't be a monstrous job.

Most programming of software is not designed as if the user is going to do everything perfectly. That's a recipe for disaster. A major part of the program that nobody sees is the reaction the application has to when things go wrong and wrong things are entered. That is the real world and it has to be catered for.

The server hardware can also be compared to your PC as it has a finite number of resources. For example your computer likes a lot of memory because when lots of things are going on everything is done by the speed of light. When there are too many computations and it can't do everything in memory it starts page-swapping and using disk drives which are slow as treacle compared to memory.

The servers on AO may just be getting overloaded and having to use the mechanical options. The overloading may definitely be due to data corruption. Worse case scenario - lots of corrupt data gets deleted to fix it.

Forgive me if what is being said in this thread is obvious to many people but surely the time has come where a brainstorming session out in the open... hopefully without wisecracks and where past feelings can be put aside for the moment should be considered. The above I'm putting out there to hopefully have people contribute whether they completely disagree or whether they have something to add.
 

·
Registered
Joined
·
3,193 Posts
It's all academic and there's little point guessing. The problem is vBulletin combined with a lack of resources, so the solutions are to upgrade one or the other. Long threads are a known performance problem on vB so bobda was right to lock them.

vBulletin's future is very murky and it wouldn't surprise me if (assuming AO is still actively managed) we eventually migrated to something else like xenForo.
 

·
Registered
Joined
·
37,649 Posts
We are not in a very well informed position so I have to agree with TB.

Were we migrated from one hardware environment to another, are the operating system versions the same, what bandwidth does the new host have, what other services are hosted on the same box, etc etc?

The list of possibilities for up-cocking is endless, and many of them seem to have been utilised to their fullest extent! ;)
 

·
Registered
Joined
·
5,473 Posts
Possible solution: Huddler


(I should point out that I have never worked for corrupt banking systems, but know a smidgen about global Internet forums and setting up enterprise solutions for a few small companies that turnover multi billion$)
 

·
Registered
Joined
·
22,760 Posts
When you have registered on AO there will be a 'parent' record set up giving you a totally unique number. For your example let's say 54321. On this parent record will be your name, avatar, email and basically parts of your profile that are unique and usually static.
It's all nom-nom, 0of0, 1of1s fault:tut:

He's been ruining his unique and usually static file format.
 

·
Registered
Joined
·
37,649 Posts
Possible solution: Huddler
From their website
Huddler Marketing blurb said:
"We migrated from vBulletin to Huddler, which is much better suited to integrate marketing and media, while maintaining and expanding on community features"
I'm a bit cynical TBH. The main thrust of the forum should be to provide the service which is being paid for by the subscribers. Bounty hunting alternative revenue streams SHOULD be a secondary consideration. :(
 

·
Registered
Joined
·
5,473 Posts
The marketing bit is there to entice the greedier forum owners. What impressed me was the stability and support that they offer. One of my sites went to them a couple of years ago.
 

·
Registered
Joined
·
11,604 Posts
It's all nom-nom, 0of0, 1of1s fault:tut:

He's been ruining his unique and usually static file format.

Thought it was that V6Jon that started it? :confused: :p
 
E

·
Guest
Joined
·
0 Posts
Discussion Starter #17
I would personally suspect a case of revenue maximization vs hardware minimization by the new owners personally. Oh and of course that Jon bloke everyone keeps going on about, he's a right one!
 

·
Registered
Joined
·
3,301 Posts
Its all a mystery to me,
I thought it was my pc,
then could it be me?
a simple Luddite
staring at a screen glaring white
where “WOT” had been,
in the corner a strange code 403
which I took to be the key
too solving this mystery
but a merry dance I was led
until so tired I hit my bed
for a quick respite,when on my return
and too my delight all was well
what was the magic spell
that put 403 to flight?
It is all a mystery to me.

John the Luddite.
 
E

·
Guest
Joined
·
0 Posts
Discussion Starter #19
Its all a mystery to me,
I thought it was my pc,
then could it be me?
a simple Luddite
staring at a screen glaring white
where “WOT” had been,
in the corner a strange code 403
which I took to be the key
too solving this mystery
but a merry dance I was led
until so tired I hit my bed
for a quick respite,when on my return
and too my delight all was well
what was the magic spell
that put 403 to flight?
It is all a mystery to me.

John the Luddite.
:lol::thumbs: Made me chuckle at my desk John.
 
1 - 20 of 23 Posts
Top