Thoughts on new authentication guidance in OWASP Application Security Verification Standard (ASVS) v4.0

[This content was originally posted in a series of tweets, but it also made sense to share it here.]

OWASP released v4.0 of the Application Security Verification Standard (ASVS) in March, listing security practices for organizations to design, code, and test apps against. There were substantial content changes in the authentication section, so after reviewing it I wanted to tell you what I thought about the changes.

The authors state in the V2 Authentication Verification Requirements section that their goal is bringing this standard closer in line with significant authentication changes published by NIST in the SP 800-63 Digital Identity Guidelines update that came out after ASVS v3.0.

There are 57 requirements in section 2 for ASVS version 4.0, compared to 26 in the same section of 3.0, which more than doubles the number of previous requirements.  Around 9 requirements are seemingly removed in 4.0. So let's go through some of the notable changes to the standard.

Requirement 2.1.1 establishes a minimum password length of 12 characters for users, which is one big divergence from NIST’s minimum of 8 chars. While this is justifiable for security, I do think it will cause protests, especially if dealing with third-party or legacy apps that have hard-coded lower length settings.

2.1.4 asks for support of Unicode characters in passwords. Another good change, and while I don’t have statistics I suspect a large number of Internet apps can’t meet this requirement today. Many of them are still struggling just to allow symbols (see @PWTooStrong).

2.1.6 talks about verifying the old password to select a new password, but removes text from older standard about new password confirmation. I suspect that with the addition of 2.1.12 saying to add a password field unmasking option that OWASP made this change for usability (see Why the Confirm Password Field Must Die).

Requirement 2.1.7 expands on previous guidance to prevent use of common/weak passwords to specifically recommend use of a 1,000 - 10,000 entry blacklist, either maintained locally or transmitted securely using a third party like @haveibeenpwned or Azure AD Password Protection.

2.1.9 says to eliminate any password complexity policy requirements or restrictions. This places responsibility on blacklists and minimum password lengths to prevent bad password choices. It's a very contentious change for orgs who've used complexity policies for decades.

Speaking more generally, the migration away from password complexity policies to blacklists is a major shift that needs additional research. If you implement password blacklisting within your org please find a way to share your lessons learned, anonymously if needed, so we can all benefit.

New requirement 2.1.8 says to provide a password strength meter to guide users towards choosing stronger passwords or passphrases. However, not all meters are created equally, so I recommend taking the time to select a good one (see On the Accuracy of Password Strength Meters)

2.1.10 instructs the removal of password expiration policies, which has gained support in recent years. But this goes hand-in-hand with requirement 2.2.1 to implement controls to combat password attacks and reduce the chances of a password compromise leading to account takeover.

Requirement 2.2.2 discourages relying on ‘weak authenticators’ like SMS and email. 2.2.4 and 2.2.7 advocate prioritizing reliance on MFA options less likely to be compromised, like OTP tokencodes, U2F security keys, or client-side certificates.

2.2.3 says to securely notify users following any changes to their credentials, emails, addresses, or if new logins to their accounts occur from a riskier/previously unknown location. The use of push notifications is preferred to better direct user attention to these events.

Requirements in 2.4 expand OWASP guidance on password storage to include aspects like salt randomness and length. But they don’t mention Argon2 or scrypt, instead offering work factor advice for bcrypt and an oddly inflated PBKDF2 iteration count of 100,000 (NIST is satisfied with 10,000).

They do discuss Argon2 and scrypt in the OWASP Password Storage Cheat Sheet, so it’s likely these documents just need to be aligned better. The PBKDF2 iteration count discrepancy may just be a typo.

2.5.2 now calls for the elimination of security questions rather than just making sure the ones in use are ‘good’. Many industries, especially US banking, still rely on these for secondary authentication. But with stronger MFA options OWASP thinks they can be phased out.

Sections 2.6 and 2.7 are brand new and describe recommended security elements for Transaction Authorization Numbers (TANs) and out-of-band (OOB) authenticators. These tie back directly to NIST guidance for designing or implementing these solutions.

Section 2.8 likewise gives guidance on one-time password (OTP) authenticator use, with section 2.9 discussing cryptographic security key authenticators (FIDO U2F). You may not develop these functions in-house but should validate that your vendor has done so properly.

Section 2.10 adds further guidance on storage of passwords or API keys used by application code, including not to “rely on unchanging passwords”. I think a better word would be “unchangeable” so they should be updated as needed but not on a set schedule.

So what was removed in section 2 of the new 4.0 ASVS? Some practices in the old standard may have been considered too basic, such as ‘require authentication for anything non-public’, ‘enforce authentication on the server’, and ‘fail securely to a default deny.’

The old 2.2 is removed which stated “Verify that forms containing credentials are not filled in by the application.” This could be understood to include password managers autofilling credentials, rather than just browsers, which OWASP may have decided to no longer discourage.

One absence is the old 2.23 practice of making sure account lockout due to login failures was flagged separately from administrative account disabling. It was a logical recommendation, so I don’t know if it was also considered too basic or whether it's removal is an oversight.

Also removed is the 2.28 guidance to ‘make sure all authentication challenge responses take the same time.’ Intended to prevent things like timing attacks that leak credential info, this may have been thought to be too difficult to implement consistently for the value it provided.

Old practice 2.32 wanted you to make sure administrative interfaces weren’t accessible to untrusted networks. This may have been removed because it becomes more difficult in a cloud-hosted world, or maybe because some apps use the same login interface for admins and normal users.  [project lead Andrew van der Stock commented "We did that because "where" is not as important as "who". The idea of Fortress Admin is laughable and always has been. Let's move beyond layer 3 restrictions."]

Finally, gone is the old 2.18 requirement to verify that username enumeration isn’t possible in login or account recovery functions. While preventing account enumeration is good, it tends to provide little value at the expense of the user experience.

I created a Google spreadsheet to compare the section 2 authentication requirements of the ASVS 4.0 and 3.0 side by side. You can access it here: https://docs.google.com/spreadsheets/d/1UbOsbgv4WsmuVuL8M3NoCRD7UQKAw7vl6BLYaLk-EtI/

Wrapping up this review, I want to thank the OWASP organizers and volunteers who developed this standard and made the tough decisions about what practices to include/exclude. Project leads include @vanderaj @JoshCGrossman @dcuthbert @m8urnett & @manicode.

When Should You Change Your Password?

May 7th is known as Password Day on the Internet.  The annual event is used to draw attention to common password problems and offer tips to people for choosing and managing passwords better.  Found among this advice is guidance to change your passwords on this day, or to at least change them regularly. Unfortunately, this practice it isn't as clearly beneficial as those offering it might hope.

The reality is that we security professionals have been advocating regular password changes, and sometimes forcing users to comply, for decades with very little evidence that it is useful.  The only reason to change a password is if we know an attacker has obtained it or if we suspect this is likely.  We simply don’t have data that shows regular password changes do make an impact on the problem of password theft.

I gave a presentation on password expiration for a technical audience at PasswordsCon last year, but it didn't address the challenge from a user's point of view.  So, I wrote this related article for the average Internet user, and not only technology or security industry veterans.

With that background, here are the questions I would ask yourself before changing your passwords.  I've listed these from the most important questions to the least important.

Has a company told you that they've had a security breach and recommended you change your password?

Usually responsible companies will alert users if they have suffered a security intrusion that put user passwords at risk.  This alert will hopefully come in the form of an email or letter to you, but might also show up as an notification when you log into their site.  Some sites force you to change your password following a breach, while others simply recommend you change it and leave that decision up to you.

On some sites your stolen password may be cryptographically protected, but this may only buy you a few days or weeks following a breach before attackers can 'crack' them and start using your account.  I recommend quickly changing your password in any situation where you've been alerted there was a security breach.

If you have used this same compromised password on other sites you should also change those passwords to a new value since some attackers will try to use your credentials everywhere they can.

Have you noticed suspicious account activity on a site?

Sometimes noticing suspicious activity with your account on a site is easy.  If you see posts on your Facebook account that you didn't make, that’s a pretty good sign something is wrong.  Likewise, unfamiliar purchases or other financial transactions on an account is a red flag.

But other times there may be more subtle clues that you’ll have to search for.  Are messages marked as read when you haven’t reviewed them?  Was the last account login time recorded when you weren't using your account?  Has your email address, phone number, or home address been updated to an unfamiliar value?

Some sites are very good about helping you notice unusual behavior, such as when someone logs into your account from a new computer or device.  These companies might send you an email to alert you of this behavior or of sensitive changes to your account.  While scammers sometimes fake these emails in ‘phishing’ attacks, you can often avoid clicking links in suspicious emails and log directly into the site to see if the warning is legitimate.

If you see reasonable signs that someone else has been using your account then this is also a good time to change your password.

Have you logged in from a shared computer or had a virus since your last password change?

Shared computers are the type you might find in a library, school, Internet/gaming cafe, or hotel.  They aren't owned by you and they typically don’t offer much reassurance that they are secure against malicious tampering.  An attacker can install software to capture any passwords you type while on the computer, and then use that information to later impersonate you.

If you want to log into a site while on a shared computers then consider changing any passwords you used once you are back on a trusted computer or mobile device.  Otherwise you might want to avoid shared computer use altogether since there’s a chance an attacker can carry out their crimes before you have time to change passwords.

If your computer or mobile device has been infected with a virus or other malware there’s also a good chance your passwords were captured during the infection.  Once you are certain that the malicious software has been removed you should consider changing any passwords that were exposed.  If you still have malicious software on your computer then simply changing your password may not help. The criminals will be able to capture your new password when you type it in and they can continue to misuse it.

If you aren't sure how long your computer was infected, and thereby what passwords are at risk, it might make sense to change the passwords of your most valuable accounts first and monitor your less valuable accounts for any sign of suspicious activity, changing more passwords as necessary.

Have you shared a password with someone who you no longer trust or who you may have trusted too much?

While you generally shouldn't make a practice of sharing your passwords, there are times where you probably will.  Maybe a significant other needs to read important messages for you while you’re out, or you share an account for a service that the whole family uses.

Problems with this habit tend to arise when either a breakup or other relationship distress occurs that changes how much you can continue trusting the person with your password.  Amidst the usual stress of these events it is still important to take the time to think about what accounts are now exposed to potential abuse and to make password changes where needed.  Even if you part on good terms, it can still be a good idea to remove the possibility of someone’s curiosity or jealousy later tempting them to get into your accounts.

Other situations you should be careful with are people who ask for your password with seemingly legitimate motives that you later begin to distrust.  For example, maybe you get a phone call from someone representing a company that claims to be fixing a problem for you but needs your password.  Or maybe a coworker or fellow student is working on a project with you and needs your password to access shared information.

In these situations you might comply with the request initially and realize after the fact that it probably wasn't a good idea.  Change these passwords and think about how to avoid these situations in the future.

Have you used this same password on another site that is much less valuable?

Another common password recommendation is that you use a unique password for every site you log into.  This recommendation it is very helpful in limiting abuse of your accounts if you choose to follow it.  However, many people share passwords between at least some of their accounts to ease the burden of memorizing a long list of secrets.

If you've chosen to share some passwords then I recommended using unique passwords for your most important accounts (banking, medical, email), and only sharing passwords for less important sites. For example, you may choose to have the same password for Facebook and Twitter if it doesn't bother you than an attacker may be able to get into both accounts without additional effort on their part.  You might also share a single password between all news or entertainment sites you visit if you don’t have any financial info tied to them.

What you should avoid is using the same password for an important site (e.g. your bank) and a less important site that may not do a good job protecting your account.  Sites that don’t deal with sensitive info typically have less comprehensive security, which can leave them more susceptible to hacking.  If an attacker steals your password from one of these sites you want to prevent them from getting into your other accounts that do contain sensitive data.

So if you've shared a password between accounts like this you probably should change the password of the more important site to something unique.

Has it been a few years since you've last changed your password?

Finally, after reviewing all these other factors, you can consider how long it has been since you last changed a particular password.  This acts as a catch-all in case you didn't notice suspicious activity, didn't remember sharing a password, didn't know you were infected with a virus, etc.

The only problem with this approach is that we don’t know how often changing a password simply based on its age is actually useful.  We have no idea whether changing a password every 6 months prevents enough hacking to justify that time frame compared to changing it every year.  Our industry has adopted guidelines, mainly for corporate passwords, but these are founded in habit rather than careful scientific reason.

So I won’t pretend that there is a lot of value in setting an arbitrary date on when a password should be changed.  If it makes you feel better to change them every few years, and you don’t mind the work, then go ahead and do it.  If you don’t want to change them regularly, and aren't required to, then I would focus that energy on running through the above questions on a regular basis.  In the long run this latter practice will probably prove to be more valuable to you.

So you've decided to change a password

To make changing your password worthwhile your new one should be something unrelated to the previous password value.  Going from “Muffins14” to “Muffins15” probably won’t protect you from an attacker committed to accessing to your account.

I would also strongly recommend that you take this opportunity to start using a password manager if you aren't already.  Password managers act as a backup to your memory so you feel more comfortable choosing better passwords and changing them as needed.  Some password managers can even supply your password automatically to the sites you log into, removing the burden on your memory entirely.

Here are a few of the more popular password managers:

Free versions:
- PasswordSafe
- KeePass

Paid versions:
- 1Password
- Dashlane
- LastPass


While you are changing your passwords look in your account settings for two-factor authentication (2FA), or any multi-factor authentication (MFA) support on the site that you may not yet be using.  MFA adds another layer of account protection, usually in the form of a one-time code texted to you, a pop-up alert on your phone, or other mechanism designed to supplement the security of passwords.  These mechanisms add another hurdle for attackers to overcome before gaining access to your account, and have the side benefit of rendering regular password changes even less necessary.

More and more sites are starting to support MFA to protect accounts, and I would encourage you to try it out and take advantage of the added security.  

Are Recent Password Guessing Attacks Tied to Devastating Morris Worm?

With recent reports of online password guessing attacks, like those Apple customers may have experienced, is it possible that the Morris worm is responsible?  Yes, I mean the Morris worm from 1988.  The one that impersonated users with weak passwords as part of an attack arsenal that allowed it to spread and overwhelm the then nascent Internet.  Since password guessing was detected shouldn't we assume this worm has resurfaced?

“But Bruce,” you protest, “you're offering no evidence that these things are in any way associated!”  And to you I reply “Well, why should that stop us from speculating?”

Ok, my claim is a bit exaggerated but it was inspired by real news from Namecheap on Monday.  The web hosting company warned customers that they had detected a credential guessing attack, which “likely” matched a recently revealed Russian password collection.  This collection, publicized by security consultant Alex Holden in August, purportedly included 1.2 billion unique credentials stolen by a hacking group nicknamed "CyberVor".  It attracted a lot of attention in the news, along with its own controversy.

So why did Namecheap think the attempted usernames and passwords were associated with CyberVor?  They don't offer a reason in their posted warning.  Furthermore, I don't believe they have a good reason.  The Russian credential cache has not been made public so there are no username or password records for them to compare to the guesses they saw.

My suspicion is that the CyberVor story was still fresh in the mind of someone at Namecheap and they made an assumption that the group's data was involved when faced with their own attack.  Maybe there was also some circumstantial evidence, such as password guesses originating from Russian IP addresses.  Regardless, there's no apparent way for them to know and there's likewise no reason to assume there is a connection.

I can forgive a company for including a seemingly unrelated statement when they aren't accustomed to disclosing details about an attack, but what really bothered me was how a few members of the media failed to question it.

An article by The Register mentioned that the Namecheap news offered "anecdotal evidence" of a CyberVor connection but seems to otherwise present the information as fact.  Infosecurity Magazine's coverage also didn't question how the two events were related, and goes on to wonder if it was the first time the Russian credentials were used in an attack against another site.

I understand that news cycle driven journalism means not being able to fact check every detail being shared, but I feel like the Namecheap claim was important enough to raise a red flag, especially at these two publications.

Meanwhile, IDG News Service did challenge how Namecheap could make this connection and actually took the time to ask Alex Holden (the only person outside of CyberVor known to have a copy of their credential cache) about the claim.  Holden agreed that there wasn't support for thinking CyberVor data was involved.  A SecurityWeek article also questioned whether timing of the two incidents was the only evidence that led to the conclusion, and apparently did try to verify the statement with Namecheap.

In reality, automated password guessing, or brute-forcing, attacks have been a threat long before CyberVor made the news.  The US Department of Defense Password Management Guidelines advised setting minimum password policy standards to combat the threat of login password guessing back in 1985.  Although since the cutting edge technology of that day required passwords to be tried over 1200 baud modems the attacks took a slight bit longer to carry out.

More recently, both web sites and other Internet services have faced increased password guessing attacks.  In 2012 the video game development company ArenaNet (who hosts the Guild Wars 2 MMO) responded to thousands of customer support tickets when criminals successfully guessed player passwords.  Last year, both Konami (PDF) and Nintendo experienced millions of login attempts, taking place over several weeks, aimed at compromising their customer accounts.

One factor these attacks all seemed to share was that the hackers behind them were leveraging usernames and passwords stolen from other hacked sites.  The dangerous and common practice of reusing passwords for different companies can mean that your site is more vulnerable to attack if you have users that also maintain accounts on less secure sites.  If one of those sites is breached and their user database is stolen (possibly with passwords stored in plaintext) it can improve the success that criminals have when attempting to guess credentials on your site.  This likelihood of success probably grows when the hacked site and yours both cater to a similar customer base.

So while it is likely that the credentials tried against Namecheap customer accounts originated from one or more hacked sites, there are plenty of more likely sources for collecting password records other than the CyberVor stash.

On a positive note, Namecheap does deserve praise for having monitoring in place to detect unusual login activity, which allowed them to quickly take steps to protect their customers' accounts.  Instead of the weeks noted above in the Konami and Nintendo cases, Namecheap personnel responded within hours of the attack.  That probably made a big difference in limiting how much damage the attackers were able to do.

Hopefully the company will continue to conduct effective incident monitoring and response just in case the Morris worm does rear its ancient head.

A Review of Real World Security Questions & Answers

Yesterday I delivered my presentation titled A Review of Real World Security Questions and Answers at PasswordsCon 13.  I have written about security questions (AKA challenge questions) before, but this presentation is based on my analysis of thousands of actual user choices that were included in hacker database dumps from three different organizations this past year.

We don't often have an opportunity to see how people are actually using security questions outside of controlled surveys, so I was thrilled to dig into the data to see what additional insights it offered about their strengths and weaknesses.  Hopefully you will find it to be a useful resource when making decisions about the use of security questions in your own organization.

You can find a copy of my presentation slides with my analysis at www.passwordresearch.com/securityqs.html.  If you would like to talk about these findings you are welcome to contact me through email or on Twitter @PwdRsch.  You can also post your comments or questions on this blog entry.

Passwords Found in the Wild for January 2013

Studying the passwords dumped on the Internet by hackers back in December provided a good opportunity for me to measure the scope of the problem. Following that experience I decided to collect and correlate some new information when analyzing password dumps from January.

Overview of Password Dumps

Last month I found 110 password dumps which met my criteria* for analysis, down from 154 in December. A few of the dumps contained data from multiple sites. There were 90 specific organizations or domains named as the source of the passwords. The remaining dumps didn't identify the source of their data, or were gathered from multiple personal computers instead of from a centralized web site.

From this collection, 40 dumps consisted primarily of plaintext passwords, exposing roughly 61,000 passwords (36% of the monthly total). Another 64 dumps primarily contained hashed passwords, adding approximately 101,000 passwords (59% of the total). Six more dumps had a mixture of plaintext and hashed passwords, accounting for 9,000 passwords (5% of the total).

Compared to 450,000 passwords dumped in December, this month's total of 170,000 passwords was significantly lower. A contributing factor to this was the smaller number of dumps, but maybe more importantly there also tended to be fewer large password dumps. This month only had two dumps containing more than 10,000 passwords, while last month had 17.

I wasn't really surprised to see this result. The number of sites that are compromised and the number of passwords they disclose will always change based on what is getting hacked that month and how large the vulnerable sites happen to be.

Over time a baseline average of 150,000 - 300,000 passwords dumped each month might emerge, but this number would skyrocket every time a large site was affected by a security breach. In June of 2012 we watched LinkedIn lose 6.5 million passwords and eHarmony lose around 1.5 million. The next month 450,000 passwords were leaked from Yahoo Voices. Just this past Friday Twitter announced a forced password change for around 250,000 users whose password hashes may have been accessed by hackers (although these have yet to be publicly dumped).

There were 40 different hackers or hacker groups claiming credit for January's password dumps, and more hackers that chose to dump their data anonymously. So even the retirement, capture, or poor motivation of any particular hacker seems unlikely to have a large impact in the flow of monthly password dumps.

The biggest deterrent to future password dumps is more likely to be improvements in the security of the code and development frameworks used by the vulnerable sites, or a widespread adoption of specific security countermeasures (e.g. web application firewalls or intrusion prevention systems). This brings us to the subject of what types of sites I found to be vulnerable today.

Sites Vulnerable to Password Dumps

In January I decided to visit all the sites named in the password dumps and gather information on the software coding language they used, the category of the function they served, and the country in which they were hosted. I hoped this might provide some further insight on whether these attacks were targeted in any way or simply opportunistic.

SQL injection attacks appear to be the primary supplier of the database dumps containing passwords. Many of the dumps include actual output from the tools (like Havij) that hackers can use to automate the extraction of database contents. This seemed pretty intuitive since SQL injection is a common web application vulnerability and one that may not require hackers to gain any other illicit access on the targeted site.

What did surprise me was that the majority of the 87 named sites targeted with password dumps were developed using the PHP programming language, as shown in the following chart.

 

Netcraft's Web Server Survey for this same month shows that 39% of all web sites (around 244 million) are running PHP. While that is a large portion of the Internet, the market share by itself doesn't seem to justify PHP sites making up 91% of the total. After all, sites using ASP and JSP can be just as vulnerable to SQL injection attacks as PHP. If I didn't know that SQL injection was the primary attack method I would suspect that hackers were exploiting some PHP-specific vulnerabilities.

A more likely explanation is that the popularity of the language has led to the rapid deployment of PHP sites and PHP-based content management systems (CMS) by people who lack an education in web application security. Even though the risk of SQL injection in PHP should be fairly well understood, some organizations still end up deploying code that doesn't implement proper security controls.

Interestingly enough, I found that one of the organizations suffering a January password dump had actually showed up previously in a hacker's report of sites vulnerable to SQL injection posted on Pastebin.com over 6 months ago. So either they never learned they were vulnerable in the subsequent months or were unable to completely fix the problem before it was exploited to dump their entire user database.

Another possible explanation is that some of these sites might be unintentionally allowing attackers to connect directly to the site database and download records that way. This should be prevented by host firewall segmentation and proper database authentication, but sites may have been deployed without these precautions. I see evidence within the posted password dump files that this is happening, but I believe it is secondary to the more popular SQL injection attacks.

While all software developers and web admins should learn about SQL Injection prevention and other secure site management practices, it appears that the PHP community needs the most help catching up.

Categories of Targeted Sites

When I visited these vulnerable sites I also assigned them a category based on the site's purpose. Most of these category labels should be self explanatory, but I'll provide a description for a few of them.

Education sites were mainly universities, although there was one primary school. Business sites were a corporate web presence that mainly published data and didn't offer online services to customers. Entertainment sites could be a discussion forum or a site sharing information on a specific topic (e.g. movies or sports).

An Online Store was a business site where customers could make purchases. This type of site would be particularly valuable to attackers looking to capture stored billing information or to order merchandise and charge it to customers. As mentioned last month, sites like this often don't make it into public password dumps because attackers can sell the account data in the underground marketplace.

Info Services are the sites that sell information as a single-use or subscription service. These sites could also be valuable to attackers, depending on what type of data they make available to customers. Finance sites are banks, credit unions, or other related institutions, but in this month's case it was a single investment firm.

The chart below shows how many sites matched each category in the January password dumps.

I didn't really have many preconceived ideas about the categories of sites I expected to see targeted. Government, Business, Medical, Education, News, and Political certainly make sense if the attacker hopes to gain media attention with their data dump. 

The Online Stores, Info Services, and Financial sites make sense if the attacker hoped to make a financial gain from the attack. Although going after these sites could also certainly just be for bragging rights.

There is probably some ratio of opportunistic and targeted attacks in this mix, but it is difficult to detect unless the hackers specifically outline their motivations in the dump descriptions.

Countries Hosting the Targeted Sites

I was able to identify the country of origin for 96 of the January password dumps. There were 35 different countries represented in that total. The top 10 countries that experienced the most password dumps are shown in the chart below.

Seven other countries tied the Philippines with two vulnerable sites each. South Africa was given a bit more attention in January than normal due to one hacker group specifically targeting organizations in that country as part of a political statement.

Another observation was that half of the vulnerable sites in India were Education category sites. They were the only country that had such a large percentage of their sites in the same category. This might indicate greater web site security problems at universities in India, or it could just be chance.

Impacts on Users of Targeted Sites

At least some users of the hacked sites are likely to experience problems as a result of these password dumps. If passwords were stored in a plaintext format then any account is vulnerable to misuse by unauthorized individuals. Even if hashed, password cracking software can produce the plaintext passwords fairly quickly for all but the stronger passwords. Whether hackers care to use these stolen credentials will depend partially on what the account can be used for and partially on how motivated they are to annoy users.

Some web sites use an email address for the username, or record the email address during the account registration process. Email addresses of users were found in 96 (87%) of the analyzed password dumps in January.

The use of email addresses isn't necessarily a problem by itself, but the reality of user password practices can result in email address leaks endangering their identities on other Internet sites. If one organization leaks email addresses and passwords this allows attackers to try those credentials elsewhere. In fact, hackers have developed software tools that automate the process of trying discovered email and password combinations against a list of popular web sites.

Password reuse makes this a real problem, although mainly for the users and not necessarily for the site that was vulnerable to the password dump in the first place. Even if a user chose a stronger password to reuse, it only takes one site storing that password in plaintext to potentially expose all of their accounts.

My suggestion to users is that passwords should never be reused across any sites where you would care if your account gets hacked. Always choose a strong and unique password for each site and then store it in a password manager if you are concerned about forgetting it.

Web sites often can't do much to prevent password reuse other than enforcing good password selection controls that might eliminate the worst of the reused passwords. However, they can make sure that they use adequate password hashing and salting techniques that make the task of cracking user passwords much more difficult for hackers. Sites should also notify users when a database breach is detected and warn them to change their password anywhere else it was used, in addition to forcing a change on the hacked site itself.

Conclusion

I wasn't able to complete my analysis of all the information provided by the password dumps in January, but I'll continue to work on it and will post new results here on the blog. In the meantime, if you have any questions or comments leave them below, or contact me on Twitter @PwdRsch.

* Study Methodology

I monitored Twitter and other sites for notices that a data dump had been publicly posted. Some data dumps contained user or customer information but not passwords. Others contained only the administrator password or the passwords of a very limited number of users. I ignored these and focused only on sources that contained passwords (hashed or plaintext) of at least a dozen or more users.

I also attempted to eliminate duplicate dumps, a practice where one hacker copies a full or partial dump from someone else and reposts it as their own. Sometimes these dumps are from the same month and sometimes they are from previous months.

In a few cases the dump poster noted that they had included only a subset of the available user passwords. While I didn't count the unposted passwords, we should assume that the attacker had access to the complete user database, which would increase the total number of passwords exposed.

When reviewing these figures keep in mind that they account only for the publicly posted data which I was able to discover. Hackers certainly compromised the passwords of other sites and kept this activity secret, or shared the data over more private channels.

Passwords Found in the Wild for December 2012

In the late 1990's when I started analyzing passwords it was much harder to find samples to review.  My password collection routine consisted mainly of begging colleagues to share data or volunteering to perform the cracking for their security assessments.  Occasionally I would get lucky and find a publicly readable password file on the Internet.  Then I could dedicate a computer for several months to cracking each password database because it would certainly take at least that long before another sample showed up.

Today I find that I am actually overwhelmed with the opportunities to gather passwords.  The raw number of Internet sites that register users and collect their passwords is huge.  Correspondingly, the number of these sites that are susceptible to SQL injection or other vulnerabilities that allow attackers to extract their user databases has also grown.  Hackers are regularly exploiting these flaws and publishing password dumps to embarrass companies, to attract attention to their causes, or to simply stroke their egos.

I decided to monitor password dumps in December 2012 to get a better idea of how widespread this practice has become.  I monitored several sources (though mainly the Pastebin.com web site) for announcements of dumps and analyzed the data posted.

Study Methodology

Snippet of Password Dump Tracking Data

Some data dumps contained user or customer information but not passwords. Others contained only the administrator password or the passwords of a very limited number of users.  I ignored these and focused only on sources that contained passwords (hashed or plaintext) of at least a dozen or more users.  I also attempted to eliminate duplicate dumps, a practice where one hacker copies a full or partial dump from someone else and reposts it as their own.

In some cases the dump poster also noted that they included only a subset of the available user passwords.  However, we should assume that the attacker had access to the complete user database, which would increase the actual number of passwords exposed.

When reviewing these figures keep in mind that they account only for the publicly posted data of which I was made aware.  Hackers certainly compromised the passwords of other sites and kept this activity secret, or shared the data over more private channels.  Brian Krebs covered the underground marketplace for the more valuable passwords in his recent blog post.

Password Dump Findings

In December I found 154 dumps which met my criteria for analysis.  A few of the dumps contained data from multiple sites.  They named more than 125 different organizations and domains as the source of the leaks.  Passwords belonged to users at businesses, governments, schools, industry groups, and discussion forums.  Some dumps didn't identify the source of their data, or were gathered from multiple personal computers instead of from a centralized web site.

From this collection, 66 dumps consisted primarily of plaintext passwords, exposing roughly 221,000 passwords.   Another 82 dumps primarily contained hashed passwords, adding approximately 222,000 passwords to the count. So while the number of hashed password dumps was greater than the plaintext dumps, the number of passwords exposed was nearly equal.  Six more dumps had a mixture of plaintext and hashed passwords, but only accounted for 6,000 passwords.

Altogether, I found that almost 450,000 passwords were publicly exposed during the month. There were 103 dumps containing less than 1,000 passwords, and 17 dumps containing more than 10,000 passwords.  About 184,000 passwords (41% of the total) came from several dumps simultaneously released as part of Team GhostShell's Project WhiteFox on December 10th.

Finding that half of the exposed passwords lack the security provided by basic password hashing is disappointing.  While some of the affected sites likely have low security requirements, storing only password hashes is a pretty standard security practice that should be followed by almost every site.

Without password hashing both the poorly and well constructed passwords are exposed during leaks like these.  A user may think their password is secure only to find that their account has been compromised due to insecure password storage that was beyond their control.

Even hashed passwords can only offer resistance against attacks once they have been stolen from an organization.  Password crackers have become faster and more proficient at trying common words, names, phrases, and other combinations of guesses that can disclose a password after it has been hashed.

Conclusion

My feelings are mixed when it comes to the results of this study.  On one hand I'm frustrated with the security vulnerabilities that continue to plague many Internet sites, and on the other hand I'm eager to see what wisdom is provided by examining these leaked passwords.  The wide variety of passwords from these different sources allows researchers like me to pick and choose the password samples that seem most interesting or likely to produce the information we seek.

So these days instead of begging for passwords I'm finding myself begging for help to sort through all the password data that is available to me.

Finding the humor in changing your password

On the lighter side of things, here's a reminder of what users may experience when it comes to changing their passwords:
http://www.washingtonpost.com/wp-dyn/content/article/2009/07/12/AR2009071202012.html?hpid=sec-metro