Evaluating a Keystroke Biometric Authentication System
In this exercise we will conduct an experiment using a biometric system found on the internet.
Because many biometric systems are available on the internet other similar experiments can easily be designed.
This exercise uses the KeyTrac System that authenticates a person by their typing dynamics.
Keytrac is one of the companies that provides an additional layer of security, often referred to as "password hardening," on entering userid/password access information.
The system uses the keystroke information from the user's entered userid and password,
and the more consistently the user types their userid and password, the better the system performs (higher accuracy).
Longer userids and passwords also helps because it gives the system more keystroke input to analyze.
For reasonable statistical analysis the exercise works best for a class or group of at least 10 people, preferably 20-30 or more.
Before performing the exercise, the following should be discussed:
This exercise consists of the following steps:
- Find a partner to pair up with, preferably one who uses a similar computer keyboard.
Each of you will try to be authenticated by the system and will also try to enter the system posing as your partner.
- Each choose a userid and password, each at least 8 characters long and easy to remember
- Practice your userid/password and your partner's userid/password at least 5 times so the input is regular and consistent
- Collecting data: the data required for this exercise consists of authentic and imposter scores obtained from all the participants using the KeyTrac system.
- Each participant tests the system to obtain scores as an authentic user and as an imposter.
- Go to the KeyTrac System and perform the following tasks:
- Type as requested your userid and password to create your profile (train the system)
- Attempt to login as yourself and record the score as Authentic Score
- Have your partner attempt to login as you on the same keyboard and record the score as Impostor Score
- Each participant should now have two scores (and the partnersip four scores)
- An Authentic Score as the correct user (yourself)
- An Impostor Score as obtained when your partner tried to get access as you
- Performing an analysis of the data: the data are entered into the prepared spreadsheet that automatically performs an analysis of the data.
- One participant, or perhaps the instructor's assistant, enters the authentic and imposter scores from each participant into the spreadsheet to obtain the results.
- Determining the performance (accuracy) of the biometric system: usually measured by the Equal Error Rate (EER) or the Performance (1-EER) of the system.
- The spreadsheet results are examined and discussed to determine whether the obtained information indicates a strong or weak biometric system.
Key ingredient of exercise:
- The prepared spreadsheet that calculates the essential biometric system performance metrics.
- It assumes the authentic and imposter scores are in the range 0-100 and higher scores indicate a better fit of the input sample against the authentic profile.
- For decision thresholds from 0 to 100, incrementing by 2, it computes FRR and FAR.
- FRR = (#authentic_scores < threshold) / total #authentic_scores
- FAR = (#imposter_scores >= threshold) / total #imposter_scores
- It then plots FRR and FAR versus the threshold, and the associated ROC curve.
Student learning outcomes:
- Students learn about biometrics as a component of cybersecurity.
- Students learn that there are two types of human traits that can be employed in biometrics -
physiological (face, fingerprint, etc.) and behavioral (voice, handwriting, keystroke, etc.).
- Students learn about biometric authentication in contrast to biometric identification.
- Students learn hands-on about a particular biometric system, in this case the KeyTrac keystroke dynamics authentication system.
- Students learn how to evaluate a biometric authentication system and the associated performance metrics: FRR, FAR, EER, Performance, ROC curves.
- Students learn how to develop a sophisticated spreadsheet to analyze experimental data.
- For example, have one or several students examine and explain the formulas used in the spreadsheet.
- Students might learn that companies exaggerate how good their systems work.
For biometric systems you can check the accuracy claimed by the company against the accuracy obtained in an experiment such as this one.