I know that this title sounds a little nasty, or broad. It was originally going to be called Sabotage, but after some thought I realised that this disaster may not be intentional. It may be ignorance, or a series of mistakes, laziness or sabotage plain and simple.
I put this one in at the third most likely disaster mostly because it is a broad subject, but that doesn’t make it any less of an issue, in fact people are the most unpredictable part of any IT network in most cases. Lets dive straight in and look at a real world situation where this was an issue, then we can cover some other possible issues.
Real World – Sabotage
First up lets take a look into a real world situation where the intention was Sabotage. It happened to San Francisco (The city). A network admin locked everyone out of a series of network devices which cause a lot of grief to try and regain control and keep the system running in a suitable fashion. Check it out here – http://www.sfgate.com/bayarea/article/S-F-officials-locked-out-of-computer-network-3205200.php
Possible – ex Employee
The first possible situation I will look at would be an ex employee, this could be someone who has been let go then returns to their desk and deletes as much data as possible, or an ex employee who had remote access and their account is left active, they log in remotely and either insert bogus data, deletes data, or steels company data for personal use.
Possible – Mistakes / Laziness
I know that these two are not always tied together but they kind of have the same outcome. Lets say for example that the person who is responsible for the backups forgets to check them (we are busy people now days), and they do not get around to it check it as we never use them anyway. Then you get a virus which encrypts all data on network drives. When you go to restore the backup, you find that the backups are rubbish. This situation happens a lot, and without regular testing you have no way to know if your backups will cover you when you need them.
So what is the cost?
We know from previous posts that for our company Quirky Imports, a days outage is about $10,000 so to save some maths we will just work on that basis as an approx cost. Lets take a look at a possibility and the associated cost.
So the person who is responsible for the backups have not checked the backups lately and a virus infection means that all over 70% of shared files have been lost. This is a difficult one to gauge because you need to work out the cost of the data that you posses. I will work it out like this.
An average document takes about an hour to create, some more, most will be less so that is $25 per document (1 person @ $25 per hour). Given that some documents cost about $25 to create but have a much higher value to the company this will probably end up being a conservative figure. (Think your MYOB database file, or similar)
Just a quick note here. If you are working on say house plans and the documents take hours to complete, then the document costs would be significantly higher (Maybe a few $100).
So if you had approx 500GB of data and the average file size is about 3MB that is about 170 000 files in total (500GB / 3MB), so to loose 70% would be approx 119,000 files. Now at $25 per document that is about $2,975,000. That is correct $2.9 million dollars worth of loss because the backups were missed.
Now most of those documents may not be required to run the business, lets say you only need about 500 files to keep things going. That is still 500 x $25 = $12,500 worth of lost files. If the documents are worth more then the cost would go up appropriately.
This number is effectively doubled as it would take about the same time to re-create the documents. Lets say that all of your staff have a photographic memory and can recreate the documents in 80% of the original time the costs would be.
- Full restore = (119,000 X 25) * .8 = $2,380,000 for recreation of all documents.
- Minimal restore = (500x 25) * .8 = $10,000 for the 500 documents.
There would also be approx 1 days loss as the issue was attempted to be resolved and sorted out ($10,000) plus if you use an external IT company, the costs associated with that aswell. (8 X $150 = $1200)
Total for not having enough time to check your backups would be
- Loss of data (500 files) = $12,500
- Recreation of data (500 files) = $10,000
- 1 day of lost productivity = $10,000
- IT company costs = @1200
Total cost would be approx $33,700. And for that you have lost over 100,000 files from your organisation.
What can be done about it?
Mistakes happen, this is just a part of life. But there are a few things that you can do to protect your organisations data against users and mistakes.
The first thing you can do is manage people correctly. I am yet to hear of case where a satisfied and happy worker decides one day to delete all of the data from a shared company folder or intentionally caused damage to a business. If you need to let someone go for what ever reason then make sure it is done in a method that protects the organisation and its data. This would most likely mean that the users account is disabled before they have the chance to cause such damage.
Lastly make sure that you have the right people for the job. I once heard of a company who most of their works worked from home (Which was all over the country as they wanted the best and they could easily work remotely), someone asked them what about productivity etc and the person responded ‘Why do you hire people you don’t trust?’
One of the biggest issues I see if incorrectly set file and folder permissions. If you protect files using NTFS permissions by only allowing people or groups access to what they need you can reduce the damage that these issues can cause significantly. This would mean that if a person decided to delete data, they could only delete the files that they have access to.
Know where your data is
Do you staff use personal phones to check their work email? what about personal computers or tablets? If required how do you recover this information? (i.e. are you allowed to remote wipe a personal device if required?) What about allowing your staff to use personal USB drives at work? These are capable of taking important data or bringing in a virus.
Check and double check your backups.
A good backup schedule is not a 2 day retention. That means that if you lost a file on Friday it is gone forever by Monday. A good backup schedule has yearly backups that are kept for a significant amount of time, monthly backups that are maintained for at least 12 months (or even better 24 months) and weekly backups that are maintained for 4 to 5 weeks. This means that there is a higher chance that in the event of this type of issue you will be able to recover quickly and with minimal data loss.
Also you need to have suitable backups. There is no use having an awesome backup schedule that doesn’t allow you to recover what you need when needed. Do you need to be able to recover specific item in Exchange or SQL? If so then your backup solution should cover this.
Also use previous versions which is built into windows. It is a life and time saver.
Double Up where possible.
This isn’t easy in a small business, but make sure that you have two people who know how to do something. So if person a is away then your backups are still checked. Check on the people who maintain these systems to ensure that the important things are being maintained such as backup tests occur on a regular basis. (This may be once every 6 to 12 months)