Facebook Breached

facebooklogoFacebook announced this evening that they have been the subject of a security breach impacting laptops used by some of their employees.  According to the Facebook statement the laptops of some of their employees were last month infected with malware when they visited a compromised mobile developer website. The compromised site hosted a previosuly unknown Java vulnerability which was used to download malware onto the laptops of the Facebook engineers. Even though those laptops were fully patched and also had up to date anti-virus sofware installed on them the previously unknown malware was able to penetrate these defences and infect the computers.

Facebook discovered the breach when reviewing their DNS logs and noticed traffic going to an unusual destination. Further investigation identified an engineer’s laptop was sending that traffic. Forensic examination of the laptop identified it had been infected with malware.  After examining the malware they were able to identify how it behaved and subsequently discovered other compromised laptops on their network. Facebook state that no user data was compromised in the breach.

Facebook also informed Oracle about the previously unknown Java vulnerability. Shortly afterwards Oracle released a patch.

The infected laptops were forensically examined and the information from them and from Facebook’s logs have been shared with law enforcement. The server controlling the malware has been sinkholed and using the data gathered from that server other compromised companies have been identified and informed. For some of those companies the first they knew about the compromise was when they were contacted by Facebook.

Some lessons we can learn from the attack are;

  • Criminals will no longer attack your systems directly but use various techniques to indirectly compromise your systems. In this case it was a waterhole attack where exploits are planted on a compromised website known to be visited by the desired target. Note at this stage we do not know whether this particular compromised website was used to target Facebook specifically. It could be this site was being used to target mobile developers in general to subsequently compromise some high value targets.
  • Having your systems fully patched with up to data anti-virus software is an important part of your defences but you cannot rely on them as your sole defences. You need to have other layers in place to protect your systems.
  • Effective log monitoring and management can provide early indicators of an attack allowing you to react quckly and effectively to the breach. Criminals will target certain group within your organisation due to the access they may have to certain data or systems. In many organisations we’ve worked with IT always have elevated privileges and admin rights to their computers. This makes them an ideal target group for criminals. I do not think it is a coincidence in this case that the criminals compromised a server for mobile developers.
  • Good forensic investigations can discover exactly what was compromised and the extent of that compromise. In many cases the response to an infection is to reformat the affected maching and reinstal the software and applications. While this is a quicker way to deal with the issue you sacrifice the ability to properly understand the extent of the breach.
  • Working with law enforcement can help in identifying who is behind the attack and disrupt their operation or indeed it could lead to arrests and convictions.
  • Sharing information with law enforcement can help identify other potential victims, who may not be aware they were compromised.
  • Providing clear and detailed information on how the compromise happened, what was impacted and what is being done to rectify the situation can provide a lot of comfort to your clients.

Well done to Facebook on being able to detect and respond to the attack. I would also commend them on the details they have shared about the incident and hopefully it can help others to learn and improve their own defences.

 

Bit9 Security Breach – Lessons Learnt

News broke on Friday evening that the security vendor BIT9 suffered a security breach. BIT9 offers a solution to clients whereby they will whitelist applications to run the PCs of their clients. This is done by digitally signing each approved application to allow it run on the protected computer. The theory behind this method is that only the whitelisted software will run on the PC. If an attacker tries to infect the computer with malware they will not succeed as it will not be on the authorised list of applications and therefore not run.

Brian Krebs broke the news that somehow attackers broke into the BIT9 network and then used BIT9’s own digital certificates to sign and push malware out onto the networks of some of BIT9’s customers. So far there is no indication as to how BIT9’s customers who were attacked detected the intrusion. It would be highly ironic if the malware was detected by anti-virus software, particularly so given this blog post “It’s the Same Old Song: Antivirus Can’t Stop Advanced Threats”.

From BIT9’s own blog post on the incident they sat “Due to an operational oversight within Bit9, we failed to install our own product on a handful of computers within our network. As a result, a malicious third party was able to illegally gain temporary access to one of our digital code-signing certificates that they then used to illegitimately sign malware”. While it would be very ironic if it was anti-virus software that detected the malware sent by BIT9, this incident is a classic example of why relying on one technology to protect your network can be so risky.

Should that technology fail then your whole security can be undermined.  This is commonly referred to a “brittle security”, a term coined by Bruce Schneier. It also highlights a phrase I have used with my clients when highlighting the trust they place with staff, partners or vendors; “those you trust the most are the ones that can end up hurting you the most”

The Bit9 breach is a classic illustration of those two statements in action. Bit9’s security was breached because of an “operational oversight” they did not manage to use their own product on all of their systems. It also shows how attackers are now using the supply chain of high value targets to attempt to breach their networks. I have no doubt that this attack, similar to other attacks such as the one against RSA in 2011, was done to leverage the trust Bit9’s customers placed in the Bit9 solution.

The lessons we should learn from this are;

  1. No software, including security software, is 100% secure (as per this old blog post). Regularly review your security software to ensure it is up to date.
  2. A layered defense, as outlined in this whitepaper I wrote for Tripwire, will help reduce the impact of a security layer failing.
  3. Ensure your security infrastructure is not reliant solely on software. Remember People, Process and Technology are the three pillars to a secure environment.
  4. Develop a patch management process that should be specific to your security software. Remember this is the software you depend on to protect your information, networks and systems so you will need to treat it differently from the accounting package deployed within your company.
  5. When selecting a product do not be afraid to question the vendor on how they manage vulnerabilities, patches and updates for their products. Also ask them how they will alert you, their customer, to security issues with their software.
  6. Your security assurances need to extend beyond your own perimeter and include your suppliers, partners and any other organisations with access to your data and/or systems that you trust and rely on.
  7. There is no such thing as 100% security. Someone or something and some particular point in time will fail and provide a potential exposure, so plan accordingly.

UPDATE 11th February

SC Magazine have covered this story with quotes from this blog post.

Likewise The Register has quoted this post in their coverage of the breach.

DarkReading interviewed me and included this post in their coverage of the breach.

UPDATE 12th February

Eric Schurr from Bit9 has commented below on the blog post to clarify how Bit9 works. My apologies if I caused any confusion in my own explanations.

Security Breach At the North Pole

Well it had to happen. Reports are coming in of a serious security incident at the North Pole resulting in the loss of Santa’s naughty and nice list.  It appears that an unencrypted USB key containing the list was lost.  There are already reports that the information on the USB key has been leaked. Obviously the repercussions of this will be felt by many around the world.

This video goes into the breach in more detail;

Santa Gets Hacked! from Twist and Shout on Vimeo.

Databreach at O2 Ireland

Today the mobile network operator O2 announced that it suffered a security breach. The breach occurred in the summer of 2011 when O2′s IT provider IBM lost a backup tape. O2 was made aware of the loss this summer an in their press release say they have been working with the Data Protection Commissioner’s office since. The press release from O2 states that the tape “had been misplaced” and that “While the tape remains unaccounted for it is possible that the tape has simply been misplaced within an otherwise secure location in O2.”

The release goes onto highlight that the tape itself was used mostly for backing up internal data belonging to O2 and that “it is possible that it could contain some personal data, it is more likely that it simply contained information about O2′s normal business affairs and company information”

After reading the release there are a number of issues that it raises in my mind, in no particular order they are;

  • Why does O2 not know what was on the tape? Most backup systems have a logfile or record of what data was backed up.  It seems strange to me that there is no record as to what data was, and was not, backed up onto the tape.
  • Why was the tape not encrypted? Copying data onto a tape means at some stage that data can be read back from the tape. This means anyone with the same type of tape drive and software can restore the data.  If that data is not encrypted then anyone with that equipment can restore and read the data. If the data is encrypted then even restoring it from tape makes it unaccessible to those without the proper access.
  • Why did it take IBM so long, nearly a year, to notify O2 about the loss of the tape?
  • Why did O2 take so long to notify customers of the potential data loss? Their press release states they were aware of the loss in July of this year, however it took 5 months to notify customers.  Under the European Communities (Electronic Communications Networks and Services) (Privacy and Electronic Communications) Regulations 2011 (SI 336 of 2011) specific obligations are placed on on providers of publicly available electronic communications networks or services to safeguard the security of their services. O2 as a telecommunications provider would come under these regulations.  In particular under the above regulations O2 is obliged in the “case of a personal data security breach affecting even one individual, providers of publicly available electronic communications networks or services must without undue delay:
    • notify the Office of the Data Protection Commissioner of the breach (even in circumstances where it considers the data would be unintelligible to third parties) including a description of the measures to be taken to address the breach; and
    • notify any individual that may be adversely affected by the breach. services

Within the press release O2 highlight minimises the breach by saying “it is possible that it could contain some personal data, it is more likely that it simply contained information about O2′s normal business affairs and company information.” So while the risk to customer data may be low it should be noted that information about its “normal business affairs” could also be highly sensitive.

As I have said before one of the important things in incident response is to learn from the incident. This applies not just to incidents in your own environment but also incidents in other organisations. The key lessons from this incident I see are;

  • Make sure you catalogue what data you back up.
  • Store those catalogue that data securely so you can reference it at a later date.
  • Have an inventory of your backup media and regular check that inventory to make sure items are not missing or “misplaced” Encrypt your backups. This should apply to all data you backup and not just data that falls under the Data Protection Act.
  • Regularly restore your data to ensure your backups are working as designed and that you can access the data.
  • Securely dispose of old backup media when no longer required.

LinkedIn LeaksOut

Earlier today it emerged that a large database of 6.5 million passwords belonging to users of the popular professional network, LinkedIn, was leaked onto the Internet.  I was first made aware of the issue early this morning by Per Thorsheim (@thorsheim), a Norwegian security professional who specialises in password security. Once the news broke on Twitter a number of people I know checked the database and confirmed that their passwords were in the file indicating that this indeed was a genuine leak.

For most of the day there was little or no communication from LinkedIn regarding the suspected breach. Many people were left to speculate was the database real or was it simply a hoax? Another question being asked was how old is the database?  Was it leaked recently or a few months ago?

Late this evening LinkedIn issued a statement confirming the breach and they are still investigating the issue.  They also outlined the following steps for those users with compromised account;

  1. Members that have accounts associated with the compromised passwords will notice that their LinkedIn account password is no longer valid.
  2. These members will also receive an email from LinkedIn with instructions on how to reset their passwords. There will not be any links in these emails. For security reasons, you should never change your password on any website by following a link in an email.
  3. These affected members will receive a second email from our Customer Support team providing a bit more context on this situation and why they are being asked to change their passwords.

For those of you responsible for the security of your organisation’s systems I would encourage you to communicate the above messages to your users but to also reinforce them with;

  • Alert all your users to the breach and if tell them if they use LinkedIn to change their passwords. As we have not yet been informed by LinkedIn of the root cause for the breach it is possible any new passwords could be compromised again. However, it is a good proactive step for users to take.  Instructions on how to change your LinkedIn password are available here http://help.linkedin.com/app/answers/detail/a_id/2873
  • If they use the same password for LinkedIn on other websites, networks or indeed your own corporate systems tell them to change the password on those systems too.
  • Some websites will promise tools to allow people to check if their password is in the compromised database. Tell users not to use these services as they have no way of validating if the authors of such tools are legitimate or are simply using it to gather passwords.
  • Remind them that if they receive emails that look like they come from LinkedIn they should not click on any links within that email.  LinkedIn have stated “There will not be any links” in any of their official emails.  It is good security practise anyway to remind users not to blindly click on links.  Below is an example of such an email sent to me earlier today, note the fake sender email address and also how a lower case “L” is used in the url to replace the I for “LinkedIn”

 As with all security breaches, even when the breach is not in your own organisation, there are lessons that can we can learn. Here are some key points I have taken from this breach;

    • Communicate regularly and clearly with the key stakeholders during a breach. Lack of communication means people are left wondering if you are aware of the issue, understand what is going on and are dealing with it.  It also allows those affected to make informed decisions on the impact the breach may have on them.
    • Let people know what actions you have taken and what you plan to do. For example, have you contacted law enforcement? Have you contacted any relevant regulatory bodies? (see my note below regarding the Irish Data Protection Commissioner)
    • What measures are you putting in place to contain the breach? What steps, if any, do users need to take?
    • Make people aware how they can contact you for more information or to alert you of a potential security breach. One of the issues many had today was not knowing who in LinkedIn to contact to make them aware of the issue.
    • Regularly monitor the Internet as an early warning mechanism to alert you to a potential breach. If you see conversations on social media sites about your organisation, or keywords related to your organisation, then this could be an early indication you suffered, or will suffer, a breach.
    • You should also monitor text and file sharing sites for items relating to your organisation, again as possible early indicators of a breach.  Xavier Mertens has a great tool called Pastemon.pl on his blog for monitoring Pastebin for such a purpose.
    • Ensure your developers understand how to securely manage passwords in their applications. Refer them to the Owasp projects guide,  the SANS Institute’s Top 25 Most Dangerous Software Errors and to the SafeCode initiative.
    • Make sure your developers know how to keep a password database secure, here is a good overview from Javvad Malik on why you need to hash your password database.

There will no doubt be further ramifications from this breach over the coming days.  The Sophos NakedSecuirty Blog says that 60% of the passwords leaked have already been revealed, over the coming hours and days the remaining passwords will no doubt suffer the same fate.

Another issue that LinkedIn will need to consider is what will their interaction be with the Irish Data Protection Commissioner’s office?  The Data Protection Commissioner has published its Data Breach Code of Practise and as LinkedIn has its European Headquarters based here in Dublin the code can apply to them.  However, note the code is not mandatory for organisations to follow.  It will be interesting to see what happens in that regard and whether or not the Data Protection Commissioner will investigate the breach.

Hopefully, LinkedIn will update us soon on this issue (at time of writing there was no further update from them) and answer some key questions;

  • How old is the database that was leaked onto the Internet?
  • Was that the complete list of compromised passwords or were more compromised? If so how big is the compromise?
  • How did the password database get leaked and what has been done to prevent it from happening again?
  • Have only the passwords been compromised or is there a corresponding list of accounts that has also been compromised? If so will those users be alerted?
  • Were the attackers targeting any specific users within LinkedIn and if so what type of users were targeted?

Finally, the old saying it never rains but it pours rings true today for LinkedIn as earlier today news broke of a privacy breach relating to the LinkedIn App for the iPad and iPhone platforms.

More resources

    • For those of you looking for more technical details into the passwords themselves, there is an excellent blog on the issue here
    • BH Consulting’s free whitepaper on developing your incident response capabilities is available here.
    • A white paper “Ten Steps for Early Incident Detection“ I developed in conjunction with Tripwire Inc is available to download here.  There are also links to a webinar I gave on the same topic.
    • Neira Jones, Head of Payment Security at Barclaycard, has an excellent post on “The Social Media Side of Incident Response
    • Finally, I talk to Javvad Malik at Infosecurity on how to deal with a security incident.

 

Security Breach at MyJob.ie

Tonight I got an email from the online recruit arm of Bond Personnel, MyJob.ie, to inform me they recently suffered a security breach and were sending me a precautionary email to change my password. While there are no details as to what information the attackers accessed or how they manage to breach MyJob.ie’s security, there are two interesting points to note;

  • MyJob.ie say they were not the primary source of the breach. This leads to the question which of their providers were breached?
  • The attackers have already been arrested and a file sent to the DPP.  If this is the case, when did the breach originally occur and why did it take so long to notify those impacted?

The other question that is of interest is what is MyJob.ie’s data retention policy for holding client data? I have not used that website for well over 10 years,  so my data would be well out of date and no longer useful.  Indeed in the Data Protection Commissioner’s report for 2008 he mentions a security breach at jobs.ie and highlights they had retained personal data of clients for “an unnecessarily long period of time”. 

If you have been impacted by this breach I recommend that you

  • You change your password for MyJob.ie
  • Do not use the same password across different systems.  If you have used the same password on different systems then change them to an individual password on each system.
  • Do not respond to any emails that may be phishing emails looking for your personal details

The text of the email is below;

Dear Honan,

I am writing to bring your attention to a recent security breach on the server hosting Myjob.ie. The breach was quickly identified, and the Gardai have apprehended two individuals who are now the subject of a file being compiled for the Director of Public Prosecutions. Although Myjob.ie was not the primary source of the breach, as a precautionary measure we would ask all users to immediately change their password. Furthermore we would ask you to observe best practice in choosing all internet passwords and do not use the same password for more than one internet service. If you do use the same password for multiple services we would strongly urge you to rectify this immediately by logging into those systems and choosing a new password. Also, please note that reputable companies do not request personal details by email, if a company contacts you do not give any personal information until you have established they are legitimate.

  • Never give out personal banking information
  • Do not share your passwords with anyone
  • Do not open email attachments if you are suspicious, especially .exe files.

Please accept our apologies for any inconvenience or distress caused by this precautionary email. Should you wish to contact us please send an email to security@myjob.ie

Yours sincerely,

John Doupe

Lulzsec Ups The Ante

There have been a string of breaches against various companies claimed by a hacking group called Lulzsec.  They have attacked organisations such as Sony, the US Senate, the security company Unveillance, the Atlanta chapter of an FBI affiliate group called Infragard, Bethedsa Software, the British National Health Service, PBS and numerous others including many pornography sites.

They claim to be highlighting how weak the security of these organisations is and to teach them a lesson in how to secure their systems.  By any logical reasoning this is not a valid argument.  If you were to equate this to real life it would be similar to someone breaking into your house and leaving a note on your kitchen table to tell you that the lock on your front door was weak and while they are at it, taking some private information and posting it on a noticeboard for everyone to see.

Lulzsec has been getting a lot of publicity with many people acting as cheerleaders as they cause havoc across the web.  Many see them as a group that is finally forcing organisations to sit up and take notice of their lax security practises and argue that this is for the greater good.  However, in most countries what Lulzsec is doing is against the law and the actions they are taking are criminal acts.  There is also the matter that in a number of cases Lulzsec has posted the personal information of the customers of the sites that were breached onto the Internet which now poses a security threat to those individuals.  There are more ethical and acceptable ways to make companies aware that their security is not up to scratch and does not involve putting innocent people at risk.

Tonight may be the time when Lulzsec overreached themselves.  It appears they launched a Distributed Denial of Service (DDoS) attack against the CIA website, www.cia.gov.  At the time of writing the CIA website is not reachable.

I suspect that they may have tried to breach the website but were unable to do so and as a result have simply blocked all traffic to the site.  This may not expose any sensitive information or breach the security of the site, but it does present a very embarrassing situation for the CIA.  This action, I am sure, will not go down well with the authorities to be and the CIA, and by extension the US Government, have a lot more resources open to them to track down the source of the attackers than say Sony or any of the other systems that they have attacked.

In addition to the CIA, Lulzsec have also drawn the ire of another infamous hacker called th3j35t3r. Th3j35t3r appears to be pro-western hacker and has been responsible for a number of attacks against websites supporting extremist terrorism.  In the tweet below he tells Lulzsec “re your last hit.  Gloves off. Expect me.”

It promises to be an interesting few days ahead for the members of Lulzsec and those of us looking on.

UPDATE 16th June 2011

Thanks to a very interesting discussion with attrition.org on Twitter a number of items have been pointed out to me;

In the third paragraph I state that there is no “logical reasoning” behind Lulzsec attacking certain companies to highlight “how weak the security of these organisations is and to teach them a lesson in how to secure their systems.”  As was pointed out to me, just because I do not agree with their methods does not mean there is no logical reasoning behind it.  This is a very valid point.  While I do not believe breaking into a system and publishing the information found there is the correct way to show how ineffective an organisation’s security is, does not mean that it is not a way to demonstrate it.  I also fully accept that the more ethical, legal and perhaps, as some would argue, naive way is not always effective as companies can, and in some cases will, choose to ignore the findings they are presented with.  But does this justify breaking into their systems and publishing their information or that of their customers?  How do we determine what is the right way in this situation?  Who or what gives individuals the right to break the law and hack into a system and expose sensitive data?

What are your thoughts on the issue? What is the most effective way to get organisations to address issues with the security of their systems without having to break the law or put innocent users at risk?

Security Breach at NUI Galway

While on twitter last night I was alerted by @_Aella to a breach at NUI Galway.  According to the information posted on the college’s website they appear to have recently been advised that a file containing the contact details of students who registered or were pre-registered to the college in September 2008.  The statement goes on to say that the breach was dues to “ a security issue with the NUI Galway Clubs and Societies computer system.”

The information accessed, while it may be annoying to those impacted, is of relatively low threat to them.  It is their personal details such as their name, student ID, phone number and NUI Galway email address.  The college assures people that “no other personally identifiable information that you have supplied to the University was at risk.” and that “we encourage you to be aware of common text/ email scams that ask for personal or sensitive information.”

The issue has been reported to the Data Protection Commissioner under the Data Security Breach Code of Practise.  It will be interesting to hear will their be any additional details released as to how the breach occured and whether this was due to an external attack or due to lax internal security controls.

Malicious Attack Against CAO Website

Monday the 23rd of August was a big day for many Irish students as their anxious wait to see if they had been accepted into their preferred third level college was finally over.  Many logged onto their computers and nervously accessed the CAO website.  However, many were ùnable to access the site as the CAO website was victim to a malicious attack.  According to a press release issued by the CAO yesterday “Access to the CAO website was affected because of a malicious attack from an unknown source this morning.  The CAO website was available intermittently between 6.10 am and 1 pm today when the problem was resolved by CAO technical staff. The system is being monitored 24 hours a day to ensure continuity of online services.”

Without hard facts on exactly what type of DOS attack it was and other details of the attack it is difficult to make any judgement on the event.  However, yesterday’s attack highlights that no matter what business your organisation is in you need to accept that once you are connected to the Internet you are a potential victim of an attack.  At IRISSCERT, www.iriss.ie, we see attacks against Irish websites on a daily basis.  Most of these attacks are by criminals targeting websites to use them to host their criminal activity, be that hosting a phishing site or spreading computer viruses.

Without the details of the attack it is hard to know what exactly happened.  DOS attacks can take various forms from flooding the network bandwidth with so much traffic you cannot reach the site, to the server not having enough CPU or memory to cope with the load, to exploiting software bugs in the operating system, website software or the web application to cause the server to become unavailable.

Defending against a DOS or DDOS attack can be difficult but some steps can be taken to reduce the risk of becoming a victim;

  • Have appropriate perimeter defences in place such as firewalls and intrusion detection systems.  Make sure these are configured properly and updated with the latest software patches and that their rules on these devices are reviewed regularly.
  • Ensure you have adequate bandwidth with burst capacity (i.e. the ability to get more bandwidth) in the event an attack happens.
  • Agree with your ISP or hosting provider that DOS defence capabilities are built into the service you are getting from them.
  • Have all the software on the system patched and up to date with the latest releases to ensure you are protected from a software based attack.
  • Make sure your incident response plans are documented and up to date with how to tackle such an attack.
  • Have key logging and alerting facilities turned on to detect such an attack as early as possible.
  • For times that are crucial and demand is expected to be high you should have extra servers, or mirrored servers in multiple locations, configured to take the unexpected load.

They are other techniques that can be used to mitigate the impact of these attacks but the bill can soon start getting higher and higher and it ends up with who has the most resources, the attacker or the defender.

I was interviewed by the RTE 9 o’clock news and the Irish Times on this matter.

Use Webmail? Time To Change Your Password

Various news media are reporting that over 30,000 email accounts belonging to users of web based email providers such as Gmail, Yahoo! Mail, Hotmail and Aol (to name a few) have been compromised.  It is unclear yet as to the exact nature of the compromise.  Some reports state that the accounts were compromised by a phishing attack.  Others state, and some of the sources I have spoken to, state the accounts were compromised as the result of a trojan or keylogger software infecting the victims machines. 

Either way if you use a webmail based service you should change your password.  Also make sure you do not use the same password across different systems because if your email password has been compromised then those other systems could be accessed by the criminals.  If you are responsible for managing the security of your organisation then consider that some of your users may use the same password for their personal email and their corporate account.  You should monitor your access logs and if you detect any suspicious activity, such as logins from countries your users are not based in, then react accordingly.  The CyberCrime & Doing Time blog have a good post on the topic which analyses how they believe the attack may have happened.

I was interviewed by both the SiliconRepublic and RTE today on this issue