Paper Highlights - A Systematic Study of the Consistency of Two-Factor Authentication User Journeys on Top-Ranked Websites

Link to paper (PDF): https://publications.cispa.saarland/3899/1/Lyastani_NDSS23.pdf

This paper from 2023 looks at how popular websites implement two-factor authentication (2FA) from a user experience (UX) and user interface (UI) perspective. The purpose was to determine the consistency between these sites since that can have an impact on whether users are able to learn about, find, and configure 2FA when they want to. The authors use a metaphor of cars where you have to figure out the braking mechanism every time you want to drive a different model, versus all cars having a standardized brake pedal found in the same location. They argue that added friction to the 2FA setup process causes users to forgo enrollment or leave the web site altogether.

They chose 85 popular websites (like google.com, amazon.com, & reddit.com) and looked at the 2FA experience for each one. The paper discusses general UX design principles and guidelines as they relate to web sites and notes that there isn’t much published guidance specific to 2FA. So this forced the researchers to create their own list of comparison factors which would allow them to methodically categorize everything from 2FA education, feature discovery, setup process, usage, and deactivation.

Commonalities found among these sites were how 2FA was named and described, where it could be found in the account settings, and that it was an optional feature in most cases -- only 7% mandated 2FA use. Of the reviewed sites 49% called it “Two-Factor Authentication (2FA)”, another 28% chose “Two-Step Verification (2SV)”, and only 5% went with the traditional “Multi-Factor Authentication (MFA)” [factor Common-Naming-and-Location].

The authors criticize that the vast majority of sites did not promote 2FA during user account setup, either waiting to nudge users towards enrollment during a later login or other security change. They observed that 73% of the sites provided at least brief information to users about 2FA before the enrollment process started, and another 15% provided a description after enrollment had started [factor Descriptive-notification]. Their premise seems to be that better descriptions may lead to more enrollments. Less of these sites (32%) provided detailed info to help users better understand the purpose of 2FA in protecting their accounts [factor Additional-Information].

Since attackers sometimes attempt to maintain access to hacked accounts by changing 2FA details and recovery emails the researchers also looked at how this was handled. They found 44% of the sites required users to verify their identity before changing 2FA settings [factor Settings-changed-verification], with only 54% informing users of changes after the fact, for instance, by email [factor Settings-changed-notification]. This seems like an area where web sites should improve to better protect and alert users to what may be suspicious changes.

Around 45% of sites allowed users to remember their device, removing or reducing future 2FA prompts from that specific system [factor Device-Remembrance]. But sites implemented this in different ways, sometimes allowing users to opt in (like ‘Remember me on this device’) and other times requiring them to opt out. Most required users to opt in.

76% of the web sites offer 2FA recovery options in case the user can’t authenticate normally (e.g. they lose their phone). Most of those also attempt to explain the importance of the recovery options to their users [factor Informed-2FA-Recovery-Options]. The most popular recovery option was one-time codes that could be printed or otherwise saved offline. Only 7% of the sites forced users to review their recovery options during 2FA enrollment [factor Enforced-2FA-Recovery-Setup].

The authors conclude by encouraging industry associations or other standards groups to formalize better recommendations on 2FA presentation and configuration for developers to rely on. This could bring about more consistency between sites and help users better secure their accounts.

This paper is a pretty dense read in areas, especially if you only have a passing familiarity with UX or UI development, but also offers opportunities to just browse through individual site findings and see what factors applied at the time of this review.

Paper Highlights: Investigating the Password Policy Practices of Website Administrators

This paper, "Investigating the Password Policy Practices of Website Administrators", was presented at the 2023 IEEE Symposium on Security and Privacy conference. But I ran across it today and thought it provided some helpful insight into why people developing or maintaining web applications chose certain password policies. The research team interviewed a small sample of 11 US-based professionals who had experience setting or managing website password policies in order to learn not just what decisions they made, but why. These weren't necessarily dedicated security team members, but more likely developers or system administrators.

A few highlights from my read:

Password composition restrictions (e.g. what characters or what length can be used) were often a result of a compatibility requirements with existing systems at the organization. Some of these restrictions affected common symbols (e.g. "&" and "?"), but others were probably extended ASCII or Unicode characters.

One organization was still limiting passwords to 16 maximum characters because of the contentious logic that 'limiting the length was necessary because users often forgot long passwords'. A couple others didn't place any limits on maximum length.

7 of the 11 respondents said they were still enforcing password expiration despite some industry guidance starting to discourage this practice. They seemed to think this provided needed protection against account takeover (ATO) from leaked or shared passwords. Those who didn't force expiration referred to their concerns that regular changes caused more user frustration and felt their systems were secure enough to withstand password attacks.

About half the participants mentioned looking either at industry standards (like NIST's 800-63B) or the practices of other large Internet sites (like Facebook or Google) for guidance on forming their own password policies. A few cited legal or industry compliance pressure forcing certain settings.

There are other interesting disclosures, like whether these organizations blocked certain passwords (e.g. blacklists) and how they decided what passwords to block. But I'd also like to hear from those of you who have been involved in this process yourselves. What steered some of your decision making?

Paper link: https://www.computer.org/csdl/proceedings-article/sp/2023/933600b437/1OXGTWy2ktq 

Paper Highlights - Do Password Managers Improve Password Hygiene?

PDF of paper: https://dash.harvard.edu/server/api/core/bitstreams/9f5f14ef-7009-46ba-9315-6ba02e625bbe/content

We’re no strangers to recommending password managers, typically because we hope that installing the software will also lead to people using strong and unique passwords.  The 2022 paper "Do Password Managers Improve Password Hygiene?" attempted to measure how closely these password practices are actually associated with the use of password managers.  

The researchers found an initial pool of around 5,000 online participants to survey about their use of password management software.  They eventually filtered this down to a much shorter list of people (n=142) who had validated their use of a password manager that included both ‘hygiene’ reporting and storage or more than five passwords.  These hygiene reports provided some details on each user’s overall password strength, reuse, and compromised status.  The researchers relied upon these reports and survey question responses to reach their conclusions about participant password practices.

Since master passwords are key to protecting access to a password manager’s data the researchers asked how participants generated theirs.  About 54% said they had generated a new password in their heads, while 35% reused a password they had already memorized.  Less than 10% reported using a random password generated by their password manager or another random process. [Q3] When choosing what should probably be your strongest secret, we really need more people opting for a strong, random password or passphrase. 

This trend of wanting to use a password manager but not wanting it to generate every password continued for many study participants.  Around 54% of the participants indicated they were more likely to create a password themselves and just let their password manager store it. About 44% said they allowed the password manager to both create and store their passwords. [Q16a]

The researchers did divide reported data between people using Chrome for password management and people using third-party solutions (e.g. 1Password, Bitwarden, etc.).  This was one area where differences between these participant groups stood out. 79% of Chrome password manager users were still choosing passwords themselves compared to 36% of third party password manager users.  Accordingly 62% of third party password manager users allowed their software to generate random passwords, compared to only 21% of Chrome password manager users. [Q16a]

This may indicate that a lot of people still want to use passwords of their own creation, possibly because they’ll remember them better, and just have the password manager as a backup in case they forget them.

One purpose of the hygiene reports included with some password managers was to provide feedback to users on their password security so that they would take action to change highlighted passwords.  But it seems that some users didn’t understand this feature.  When asked to identify one or more reasons why they still used passwords identified as weak or reused, 35% said they were not previously aware of that classification.  Around 36% said they were overwhelmed by the amount of work needed to replace these passwords.  And 35% responded that they just hadn’t gotten around to replacing them. [Q10]

Even fewer participants seemed to know when their passwords had been reported as compromised, with 52% indicating they weren’t aware they had been exposed.  The popular reasons for not replacing these passwords were similar to the reasons they had for not replacing their weak or reused passwords. [Q12]

Password managers can only do so much to encourage password changes, although some have implemented features aiming to speed up the process for select websites.  This challenge isn’t likely to become much easier unless the web adopts a standardized mechanism for automating password changes that password managers can then implement.  It also seems hard to motivate users to care more about changing their bad passwords. A different study in 2024 found only slight improvements in password changing behavior after implementing nudges to convince users to do so.

The researchers for this paper do note that password weakness or reuse are not necessarily indicators of users making bad decisions if these issues only affect low value accounts.  Participants were asked why they thought it was okay to have weak or reused passwords and 49% confirmed that they didn’t feel these accounts were worth protecting better.  Another 40% said they needed these passwords so that they could remember them without their password manager. [Q9]

Participants who were screened out due to not using a password manager (n=1,315) were asked why they didn’t use one. When offered one or more options 58% selected that they were concerned someone else could access their computer or device storing the passwords. Another 46% were worried that malicious software might compromise their device and also their passwords.  28% indicated that they distrusted developers of password management software with their passwords. But they don’t indicate if this is because they suspect the developers themselves of malicious intent, or suspect them of being unable to properly secure the software against attack by others. [Q2]

This report includes more feedback relating to people's use of password managers, and I’d encourage you to browse through the paper to find more interesting data points on your own.

PDF of paper: https://dash.harvard.edu/server/api/core/bitstreams/9f5f14ef-7009-46ba-9315-6ba02e625bbe/content