This is mostly where we find the points of entry. Usually it’s a lack of security posture or complacency in terms of keeping things ship shape.
In general, anytime, anyplace, have laptop will travel.
For the purpose of this post, let me introduce Sandy who is the office manager for a client we recently helped. I asked Sandy about how they managed their IT before…
“Mostly you’re concentrating on business, so my IT awareness started to recede and gradually dropped below the radar. Microsoft updates still pop up, some users clicked the popup and installed updates, some do not and others are a bit hit and miss. Other update notifications appear as does organically added software over time, Norton, Adobe Reader, various art packages, social media apps, file sharing utils, smaller utilities, tested and then forgotten or just used once or twice then left.”
“The server is another job to manage, typically just left in the corner, maybe an update when someone logs on to install a Sage update, or to install a printer driver for a new company printer. Windows server updates may well nag with a popup and these are installed when seen, but not checked on a regular basis. We change the backup tapes every day, that’s about it”
“The internet connection has never caused a problem, so really, it’s never tested, likewise the firewall, never needed to make changes so never logged into it for anything until recently, I wouldn’t know what I was looking at anyway”
“The Switch, the little box with lots of lights that blink and have a lot of cables plugged into it, never had a problem with it before.”
“Wi-Fi Access point is used by people for laptops and tablets and some mobile phones, mostly just to allow devices update email and calendar events and also because Facebook is nice to have at lunch time.”
It was fair to say, that Sandy wasn’t an expert, but that’s not her job role, and changing backup tapes, and running user directed updates is about the limit of her expertise here.
The company is a small recruitment firm, based on the outskirts of London. Like most recruitment companies, they need access to the internet and use phones a lot. They also send and received a huge amount of email for which they use a hosted mail service hosted by a well-known multi-purposed provider.
They have a small number of staff, 14 to be exact, and have been taking advantage of the digital age to reach clients and candidates alike.
And this is where this starts to get interesting.
Recruitment Co. is constantly searching for new candidates and like most other recruitment companies, have their own database as well as paying subscriptions to access the CV search tools online that most people would use to submit a CV for a job they see online. They also have a client list that they work constantly to find contract placements for. As a result of the amount of email traffic they are sent and receive a lot of documents, CV’s, cover letters, requests for information, contact revisions etc.
The following is a description of events as best we can make out due to the fact that the infrastructure wasn’t geared up to record or log much information….
Somehow a document was sent to a recruitment consultant that was weaponised with a RAT (Remote Access Tool) and the email hygiene service (email scanner) didn’t catch it. When the consultant opened the document attached to an email asking for the CV to be included for a real job application this consultant happened to working on, the local Anti-Virus scanner also didn’t pick up the document as containing the malware and after a brief moment opened what looked like a really poor CV. The consultant wasn’t aware, but the RAT was installed silently in the background and sent a message back to its command and control server in the Ukraine announcing that it had successfully installed itself on a target workstation and included some information like local network details, external IP address, username logged in, number of drives and a brief network scan as well as a copy of the users internet search history and recently opened documents.
As best as we can guess here, the bad guys would have been monitoring activity on the infected workstation to work out when the best time was for them to start their tasks. As no one in the office suspected anything until much much later, it’s difficult to tell how long they had access to the infrastructure, but we can hazard a guess that they had at least 3 months’ worth of unfettered access to this one local workstation from which to work. We found some evidence on other workstations that key loggers had been installed, but we can’t say for certain that they originated from this particular set of bad guys.
Recruitment Co. (not their name obviously, but we promised to keep them out of the public eye) users started to complain about odd things happening on the workstations they were using. Older documents being left open on a user’s screen when they logged in first thing in the morning, or documents printed that no one knew or owned up to printing.
Some users also commented that things were just getting slower, accessing websites now took noticeably longer to load and sometimes the pages failed to load completely and took several attempts to work. They also started to receive far less emails than they normally would have from candidates and clients they normally had a good level of communications with. AS recruitment consultants are naturally a chatty lot, it didn’t take long for them to find out that their emails were being quarantined or junked on a lot of their client and candidate mail systems and thus started their long and frustrating discovery about how far down the hole they had sunk.
John who is one of the recruitment consultants was talking to one of his clients on the phone about some candidate CV’s that he sent over that morning when his screen suddenly blanked out. Thinking the screen had gone into screen saver mode he moved the mouse a little yet nothing happened. Then suddenly his machine rebooted, to which he made a comment about it to his client who suggested he get a coffee and call him back when he got it rebooted.
As john put the phone down, his colleague next to him also commented his machine had just locked up and he had to switch it off and switch it on again.
Then they both had the same message appear on their screens;
Sitting back in shock, john and his colleague took a minute to digest this and then closed the browser. Opened their email and continued their day thinking it was another of those annoying popups.
It was only about an hour later when they were searching for some documents they created a few days before that they realised there was a much more serious issue. Searching their documents drive on their server in their client folders, they discovered every document had been renamed and when they tried to open it their Word application refused to open it.
Other people in the office who hadn’t had this message appear also could not open the documents in other locations.
The office had become victim to cybercrime, their files, locked, encrypted and held to ransom.
This is where we came into the picture, after a quick call and some questions and answers, one of our engineers made their way to their office and was onsite in just over an hour. It took about 40 minutes to check the network and cleanse the workstations, but the files were still encrypted.
The server was sanitised and a new area prepared to restore the files from the previous night’s backup into the new area. This is where a second problem struck. The backups hadn’t been monitored, and were not set to a root level backup, but to specific folders way back when. Organically, they had grown outside of this area and the overspill wasn’t included in the backups.
Fortunately, we were able to restore about 80% of their total file set completely, with the remaining 20 % spread between locally encrypted “My Documents Folders”, Desktop and the overspill directories on the server.
In a corporate environment this would never have happened, but as it often happens, awareness of how essential IT has become in today’s working environment and how security plays a major role was overlooked. The cost? Half a day’s downtime, and a few hundred files lost for good. Those files were CV’s of candidates, pitches and financial data that resulted in over roughly over 800 man hours lost.
Lessons learned from this event;
- Monitoring the backups and doing regular random restores is essential as the current generation of ransomware is nearly always perfect in its execution of the payload.
- Enabling Shadow copies on the server with enough retention time to cover immediate and long term restores is also now essential.
- Enforcing user data redirection to the server should also have been enabled by group policy, this would have helped save the My Documents data and desktop data the users lost.
So now Recruitment co. have effective monitoring in place, they have also beefed up their mail hygiene, as well as their Anti-virus desktop deployment and more importantly, they have had their backup strategy completely rewritten. Being a small organisation they did not have the throw weight of a large corporate, so we instigated some policy controls on the network and changed the way in which their infrastructure works slightly, changed how users are able to control their local workstations (now they do not have local admin rights which is a strict no no in today’s technology). The backup strategy we moved in a radical direction from what they were used to, they still had local ( server based ) copies of the files and tape drive backups, but we also enabled a cloud based backup using Microsoft Azure backup to retain long term backups in the cloud as a further precaution. Eventually they will migrate to SharePoint online and enjoy a better experience as well as having some excellent security protocols in place to protect them in the long term.
The final change we made was to engaged with a web screening technology that we have found to be extremely successful in defeating all drive by and web based attacks. This essentially protected the users from any web based attacks like Banner adverts that are booby trapped or scripts that rely on the user to activate ( click them ) to become infected.