Post by roger on Jun 3, 2008 15:46:12 GMT
Hi everyone, it's good to be back!
Most of you will have seen that this forum has been unavailable since the latter part of Saturday, 31st May. I will outline the reason, the solution as we understand it, and our proposed course of action to minimise any long-term consequences.
Reason for Network Outage
ProBoards host their forums at The Planet who are based in Houston, Texas. On Saturday afternoon, a transformer exploded outside their Data Centre. A wall of the building was severely damaged. Nobody was injured. The fire department insisted that power to the relevant sector of the Data Centre be turned off as a precaution but we have since learned that there is severe damage to the cabling. This caused approximately 7500 servers to go offline, 32 of which are used by ProBoards. We were assured that no servers were damaged and that no data has been lost but were advised that it may be several more days before everything is fully restored.
The staff at The Planet and at ProBoards have been working literally round the clock in an attempt to restore all sites and forums as quickly as possible.
The remainder of this post was prepared two days ago and much of it is no longer relevant but I will post it "as is" so that you can see what difficulties they, and we, were in.
The Solution
ProBoards have acquired and installed 24 new servers. They then transferred data from their back-up to those new servers. Unfortunately, the back-up only includes data up to the end of April. Consequently, all posts made throughout May do not yet appear on the forum. Furthermore, the accounts of those members who registered during May will be temporarily unavailable. You are therefore seeing the forum exactly as it appeared at the end of April. We are assured, however, that everything will be fully restored in the next few days.
In order to achieve that, ProBoards now have to transfer all data from the old servers to the new ones. Only then will the forum be fully up to date. This transfer cannot begin until power is fully restored to the old servers. It is likely to be a few more days until that can be completed.
Our Plan of Action
If we allow posts to be made immediately, we fear that, once the May data has been transferred, we may end up with posts in every thread appearing out of sequence, thus:
We also fear that anyone who registered an account during May might register again in the belief that their first account has been lost. It hasn't. It should be available in the next few days.
It is also possible that threads may be duplicated once the final transfer of data is completed.
In an attempt to minimise any duplication and non-sequential posts, Dave and I agreed to put the forum into Maintenance Mode as soon as it came back online. During this time, neither members nor guests were able to view any part of the forum. This was only a very short-term measure (perhaps an hour or two). Whilst the forum was in Maintenance Mode, I disabled all further posting throughout the forum (except for this thread) and all further registration. I then brought the forum back into view - but it is literally that, VIEW ONLY.
As soon as we are satisfied that all data has been fully restored, I will enable registration and posting throughout the forum. We don't yet have a timescale for that but I hope it will be within the next 2-3 days.
At that point, this thread will be moved to the Technical Support board and this Temporary Board will be deleted.
Please feel free to discuss any of the above by replying to this thread. If you have any questions, we will endeavour to answer them.
The staff at The Planet and at ProBoards apologise for the interruption to their service which was clearly caused by circumstances beyond their control. I can assure you their efforts to restore all forums as quickly as possible has been truly magnificent and their regular updates have been very reassuring.
On behalf of HWI, I must apologise to those of you who were unaware of the reason for the forum's disappearance. I gave updates on the AAI forum (which was unaffected) in hope that some of you might see them and Dave put a link on the front page of the main site to that thread. My thanks to Joe for outlining the problem on his Cottars forum and to Stephany for answering enquiries on Facebook. Other than that, we had no practical way of conveying information to you.
I assure you this forum will be fully operational as soon as possible. Meanwhile, thank you for your patience.
TUESDAY 3RD JUNE UPDATE
It appears that the forum is fully restored and that all posts and memberships are correct as of the time immediately prior to the explosion. I have noticed some avatars and forum images are missing but I am sure we can sort those out in due course if they do not reappear automatically.
So, let's celebrate by making this forum as active as possible.
Roger
Most of you will have seen that this forum has been unavailable since the latter part of Saturday, 31st May. I will outline the reason, the solution as we understand it, and our proposed course of action to minimise any long-term consequences.
Reason for Network Outage
ProBoards host their forums at The Planet who are based in Houston, Texas. On Saturday afternoon, a transformer exploded outside their Data Centre. A wall of the building was severely damaged. Nobody was injured. The fire department insisted that power to the relevant sector of the Data Centre be turned off as a precaution but we have since learned that there is severe damage to the cabling. This caused approximately 7500 servers to go offline, 32 of which are used by ProBoards. We were assured that no servers were damaged and that no data has been lost but were advised that it may be several more days before everything is fully restored.
The staff at The Planet and at ProBoards have been working literally round the clock in an attempt to restore all sites and forums as quickly as possible.
The remainder of this post was prepared two days ago and much of it is no longer relevant but I will post it "as is" so that you can see what difficulties they, and we, were in.
The Solution
ProBoards have acquired and installed 24 new servers. They then transferred data from their back-up to those new servers. Unfortunately, the back-up only includes data up to the end of April. Consequently, all posts made throughout May do not yet appear on the forum. Furthermore, the accounts of those members who registered during May will be temporarily unavailable. You are therefore seeing the forum exactly as it appeared at the end of April. We are assured, however, that everything will be fully restored in the next few days.
In order to achieve that, ProBoards now have to transfer all data from the old servers to the new ones. Only then will the forum be fully up to date. This transfer cannot begin until power is fully restored to the old servers. It is likely to be a few more days until that can be completed.
Our Plan of Action
If we allow posts to be made immediately, we fear that, once the May data has been transferred, we may end up with posts in every thread appearing out of sequence, thus:
- All posts up to the end of April, (the back-up point), followed by
- A few days in early June, (made since the time that the forum was partially restored but before the transfer of the May data is completed), followed by
- All posts made in May (once data has been transferred from the old server), followed by
- Posts made from then onwards.
We also fear that anyone who registered an account during May might register again in the belief that their first account has been lost. It hasn't. It should be available in the next few days.
It is also possible that threads may be duplicated once the final transfer of data is completed.
In an attempt to minimise any duplication and non-sequential posts, Dave and I agreed to put the forum into Maintenance Mode as soon as it came back online. During this time, neither members nor guests were able to view any part of the forum. This was only a very short-term measure (perhaps an hour or two). Whilst the forum was in Maintenance Mode, I disabled all further posting throughout the forum (except for this thread) and all further registration. I then brought the forum back into view - but it is literally that, VIEW ONLY.
As soon as we are satisfied that all data has been fully restored, I will enable registration and posting throughout the forum. We don't yet have a timescale for that but I hope it will be within the next 2-3 days.
At that point, this thread will be moved to the Technical Support board and this Temporary Board will be deleted.
Please feel free to discuss any of the above by replying to this thread. If you have any questions, we will endeavour to answer them.
The staff at The Planet and at ProBoards apologise for the interruption to their service which was clearly caused by circumstances beyond their control. I can assure you their efforts to restore all forums as quickly as possible has been truly magnificent and their regular updates have been very reassuring.
On behalf of HWI, I must apologise to those of you who were unaware of the reason for the forum's disappearance. I gave updates on the AAI forum (which was unaffected) in hope that some of you might see them and Dave put a link on the front page of the main site to that thread. My thanks to Joe for outlining the problem on his Cottars forum and to Stephany for answering enquiries on Facebook. Other than that, we had no practical way of conveying information to you.
I assure you this forum will be fully operational as soon as possible. Meanwhile, thank you for your patience.
TUESDAY 3RD JUNE UPDATE
It appears that the forum is fully restored and that all posts and memberships are correct as of the time immediately prior to the explosion. I have noticed some avatars and forum images are missing but I am sure we can sort those out in due course if they do not reappear automatically.
So, let's celebrate by making this forum as active as possible.
Roger
Thanks to Roger & Dave for keeping us so well informed over this difficult period and of course to the Proboards people who have obviously worked so hard to get things back to normal.
It was a great relief to have AAI as a temporary meeting place so it didn't feel as bad as it might have done without anything! 
Can I have mine back please? 
