Authors: Johnathon P. Ehsani, Rebecca A. Weast, Theresa J. Chirles, Andrew Hellinger, Gayane Yenokyan, Yifan Zhang
Categories: Research Article, Teenage drivers, learner permit, logbook, performance-based graduated driver licensing, smartphone
Source: AJPM Focus
Keywords: Teenage drivers, learner permit, logbook, smartphone, performance-based graduated driver licensing
In most U.S. states, individuals aged 16‒17 years who are learning to drive are required to complete a minimum amount of practice driving (typically 50 hours) under the supervision of a licensed driver (usually a parent) before they are eligible to take the driving test.^1^ The rationale for this supervised driving requirement is that teenagers will develop skills that could provide a safety benefit during early independent driving when their crash risk is highest.^2^ Certain jurisdictions, such as Maryland and Virginia in the U.S. and New South Wales and Queensland in Australia, require that supervised practice driving hours be recorded in a paper logbook, signed by a parent or guardian, and submitted to the licensing authority as evidence that practice was completed before taking the practical driving test.3, 4, 5 Previous research with parents from Queensland, Australia found that paper logbooks provided a number of benefits, such as encouraging learner drivers to better structure their driving experiences.^6^ Some learner drivers indicated that they falsified paper logbooks by rounding up hours of practice driving and fabricating trips that were not taken.^7^ This suggests that paper logbooks may not reflect an accurate record of practice, undermining their intended purpose and creating a system that is open to misuse.^8^
Several states also endorse smartphone apps as an electronic alternative to a paper logbook.9, 10, 11 Over 95% of teenagers in the U.S. have access to a smartphone,^12^ and electronic logbooks may offer some benefits over paper logbooks, such as the provision of an automatic summary about each drive (distance, time) and the calculation of accumulated supervised practice driving hours. Eventually, they could form part of a licensing system where practice requirements are verified, allowing states to move toward performance-based graduated driver licensing.^13^
To the best of our knowledge, no studies have compared the use of a paper logbook with that of electronic logbooks to measure supervised practice driving. The purpose of this study was to measure the impacts of the logbook format (i.e., paper versus electronic smartphone app) on self-reported learner-driving behavior. Furthermore, among those who used the electronic logbook, we compared the agreement between their survey-reported trips and electronic logbook‒recorded trips and their perceptions of using an app to measure practice driving.
This study required the participation of teenage drivers aged 15–19 years and one of their parents or guardians residing in Maryland. All eligible teenagers were expected to hold their learner permit for at least 6 months after their participation in the study began. Participant dyads (teenage driver and parent/guardian) were required to speak English, have access to the Internet, and use a smartphone for <4 years using either the Android or iOS (Apple) operating system. Maryland requires drivers aged ≤25 years to practice a minimum of 60 hours under the supervision of a licensed adult during the learner permit stage. Novice drivers aged ≤18 years are required to hold a learner permit for at least 9 months, and novices who are aged 19 years are required to hold a learner permit for at least 3 months.^3^ The Johns Hopkins School of Public Health IRB approved the study.
All participants were required to complete surveys at baseline and at 6 months. Parents/guardians also completed monthly surveys about the amount, consistency, and variety of supervised practice driving completed by their teenagers.
Electronic logbook participants were required to use a custom-built app to measure their driving behavior for the duration of the study. This app measured trip length (using time stamps from the start and end of each trip), gravitational forces, and driving routes using the device's GPS. Participants assigned to use the paper logbook were instructed to complete the logbook as required by the state licensing agency. Details about the app and the study protocol are described in a previous publication.^14^
For the baseline survey, multiple scales were administered to the parents/guardians and separately to the teenagers. Baseline survey instruments included personality scales,^15^ measures of the parent‒teen relationship,^16^ teenagers’ risk-taking behavior and risky friends,^17^ previous driving experience*,*^18^ and parents’ perception of their teen's readiness to drive independently.^19^ Parents/guardians were also administered a monthly survey regarding (1) the amount of practice driving (trips and hours) and (2) the variety of practice driving completed by their teenager.
During the 6-month follow-up survey, parents and teenagers who were assigned to the electronic logbook group were asked about their perceptions of the app. Participants were asked (1) whether they would recommend the app to a friend, a measure of user satisfaction developed by Reichheld^20^; (2) whether they would be willing to use the app over a longer period than the current studies; (3) whether they experienced issues with the app; and (4) whether using the app was a burden in any way.
Participants using the electronic logbook were required to manually press the Start button to begin data collection and the Stop bton at the end of each trip. After the trip, the app presented summary information, including trip length and duration. To minimize the possibility of invalid trip data collection, such as inadvertently starting a trip or forgetting to end a trip, a valid trip was defined as having a distance >0 miles and a duration of between 2 minutes and 120 minutes.
For survey data, electronic and paper logbook participants were compared on demographic and driving characteristics at baseline using chi-square tests and Fisher's exact tests for categorical variable comparisons and t tests and Mann‒Whitney U tests for continuous variable comparisons. Mixed effects models were used to account for repeated individual measures across follow-up surveys. Negative binomial regression models with random intercepts for each respondent were used to model the number of trips and practice variety. A linear regression model with random intercepts was used to model the number of hours driven, and a bootstrap was applied to obtain model estimates. All models included logbook type (electronic versus paper) and time point (5 indicator variables representing follow-up Months 2–6 versus Month 1) and an interaction term between the study group and time point.
All models adjusted for days between the permit date and study start date, the teenager's previous driving experience, and the parent's perception of their teenager's readiness to drive at baseline, all standardized (i.e., rescaled to have a mean of zero and an SD of one). Additional models with 3-way interactions were previous driving experience or readiness to drive at baseline, study arm, and time point. Interaction terms not significant at the 0.05 level were dropped from the final model. Analyses were carried out using R, Version 4.0.2.
Regarding missing data and sensitivity analyses, when responding to monthly survey questions about the amount of practice driving, some parents indicated zero trips taken but a non-zero number of hours driven (8.2% of all responses). In cases where the number of trips was reported to be zero, the number of hours driven was also set to zero. In addition, 1 parent reported 98 practice driving trips taken in 1 week. This observation was excluded as an outlier. Sensitivity analyses showed that neither adjustments to the number of hours driven nor the inclusion of the outlier changed the pattern of results.
Agreement between self-reported and electronic logbook-recorded driving. Self-reported driving during the previous week was collected during monthly surveys. Electronic logbook‒recorded trips for the same week were identified for each participant. The Bland‒Altman method was used to assess agreement between self-reported and electronic logbook trips, 95% CIs for the difference (i.e., limits of agreement [LOA]),^21^ and 95% CIs for the upper and lower LOA while accounting for repeated measures.^22^ Comparison analyses were performed for all valid trips. Twenty-three teens who did not record trips using the electronic logbook for the study period were excluded from the analysis.
A total of 917 individuals responded to in person and online recruitment efforts, of which 337 dyads fulfilled the eligibility criteria and were enrolled in the study. Of these, 186 dyads were assigned to the electronic logbook group (intervention), and 151 dyads were assigned to the paper logbook group (control). Participants were considered to have complete data if the parent and teenager completed the baseline survey and at least 1 monthly survey. To limit differences because of geographic location, we excluded 32 participant dyads who resided outside of the state of Maryland. Four dyads were removed from analysis owing to data entry errors (3) or missing data (1) on surveys. The final sample was 147 dyads in the electronic logbook group and 131 dyads in the paper logbook group.
Individual characteristics of the teenagers and parents in the electronic logbook and paper logbook samples were not significantly different from one another (Table 1). Teenagers using the paper logbook reported higher prepermit driving and also held a learner permit longer than the electronic logbook group at enrollment. Table 2 presents the number of participants in the electronic and paper logbook groups that completed the surveys each month.
For the entire sample, participants reported driving 2.5 trips and 3.0 hours in the previous week. There was wide variability in practice driving between participants shown by the CIs in Figure 1. Hours of practice driving did not change across the study period. The number of trips decreased at Months 3 and 4 (ratio=0.87, 95% CI=0.76, 1.00, p=0.05 and ratio=0.83, 95% CI=0.72, 0.96, p=0.01, respectively) relative to that at Month 1, and practice variety significantly increased at Month 6 compared with that at Month 1 (ratio=1.12, 95% CI=1.02, 1.22, p=0.02).
Figure 1 Number of self-reported trips, hours driven, and practice variety in the paper versus electronic logbook groups by month (n=278)Note: For each follow-up month, the circles are the mean number of trips, and the lines represent 1 SD.
Across the 6-month study period, there were no statistically significant differences in any measures of self-reported practice driving (trips, hours, or variety) by logbook type (electronic versus paper) (Figure 1). Table 3 presents the estimates of the mixed effects model for hours, trips, and variety. In the first month of the study, the electronic logbook users reported more trips than the control arm, and the difference was significant (ratio=1.26 trips, 95% CI=1.00, 1.60, p=0.05). In the remaining months, self-reported practice driving was not significantly different by logbook type.
Across the 6 months of the study, there was perfect agreement (i.e., no difference between self-reported and electronic logbook‒recorded trips) for an average of 30.7% of trips, ranging between 28.4% and 33.0% (Figure 2A). When the agreement threshold included an absolute value difference of 2 trips (e.g., survey report of 6 trips and electronic logbook record of 4 or 8 trips), the agreement was for 72.5% of the sample. Across the 6 months, this ranged between 68.6% and 79.0%.
Figure 2 (A) Agreement between self-reported and electronic-logbook-recorded trips. (B) Bland‒Altman plot of self-reported and app-recorded trips by electronic logbook users (n=124)*.Note: Percentages describe the agreement between the number of trips (±2 trips). Excluded from the analysis were 23 teens from the electronic logbook group who recorded no practice driving trips during the study period.
A higher number of trips in the previous week were reported in the surveys than in the app-recorded trips (mean discrepancy of 0.99 trips). In Figure 2B, the orange line shows the mean of all paired differences. The green lines represent the LOA, which form the interval in which 95% of differences will fall. The dashed blue lines represent the 95% CIs of the LOAs. The calculated LOA for the number of trips were –3.81 (95% CI= –4.83 to –2.88) for the lower bound and 5.79 (95% CI=5.01, 6.47) for the upper bound, indicating that most discrepancies are from 4 fewer self-reported trips to 6 more self-reported trips compared with the app-recorded trips. The same patterns were observed for hours of practice and practice driving variety (data not shown).
Self-reported satisfaction with the electronic logbook was high, with 93.0% of teenagers and 91.9% of parents stating that they would recommend the app to a friend. A slightly lower percentage of participants stated that they would be willing to continue to use the app beyond the study period (89.5% of teenagers and 86.2% of parents). Slightly less than half of the teenagers (42.1%) and one third of parents (30.1%) reported issues using the app, and a small percentage of participants stated that the app was a burden (6.1% of teenagers and 4.9% of parents).
The purpose of this study was to measure the impact of the logbook format (i.e., paper versus electronic smartphone app) on self-reported supervised practice driving behavior. Furthermore, among those who used an electronic logbook, we compared the agreement between the survey-reported trip with electronic logbook‒recorded trips and asked about their perceptions of using the app. Consistent with the findings of a number of previous studies, we found that practice driving was highly variable during the learner period across the entire sample.^23^ However, we did find in the first month of the study that the electronic logbook users reported more trips than the control arm. This short-term effect on driving behavior may have been related to the novelty of the app.
Among those using the electronic logbook, the number of self-reported trips was higher than in the electronic logbook‒recorded trips by approximately 1 trip in a given week. However, the 95% CIs for the differences were wide, ranging between 4 fewer trips to 6 more self-reported trips. This suggests that the self-reported and app-recorded trip measures should not be used interchangeably. Although self-report driving measures are known to be affected by recall bias,^24^ further research is needed to establish whether the electronic logbook could be considered the more accurate gold standard measure.
Although approximately one third of participants experienced problems with the app at some point during the study, most of both parents and teenagers stated that they would recommend the electronic logbook to a friend and that they would be willing to continue using the app beyond the study period. It is not possible to state whether these usability perceptions are specific to the app used in this study or would be more widely generalizable to all electronic logbooks that measure practice driving. These findings suggest that electronic logbooks may have high levels of user acceptance, even if problems are encountered during use.
This study used intensive survey and smartphone-based measures from a sample of parent‒teen dyads to prospectively collect data on learner-driving behavior for 6 continuous months. Despite some attrition, the final study sample represents one of the largest studies of learner drivers’ behavior in the U.S. Because recruitment for this study occurred during the first months of the COVID-19 pandemic, we had to relax the inclusion criteria when in-person enrollment moved online. Although enrollment was being conducted in person at the Maryland Motor Vehicle Administration, participants were enrolled on the day they received a learner permit. After recruitment shifted online, we could no longer verify participants’ time from the permit but required that participants drive with a learner permit for a minimum of 6 months. Although this eliminated the possibility of measuring driving behavior from the same time point for each participant, previous studies have shown that practice driving often occurs before the permit period begins,^18^ and the amount of driving during the learner-driving period is relatively stable.^23^
Direct comparison between electronic and paper logbook groups was done using self-report survey data. Although the self-reported driving behavior of the participants using the electronic logbook could be compared with the trips recorded by the app, the paper logbook participants’ driving behavior could not be verified. The electronic logbook participants were required to engage with the app at the beginning and end of each drive, whereas control group participants recorded their drives to their paper logbook at their convenience, introducing the possibility that logbook app users’ required regular engagement with their driving could have altered behavior. The temporary increase in the number of trips among electronic logbook users in the first month of the study may suggest that an interactive, engaging app may be effective in altering practice driving during the learner period.
Electronic logbooks that can verify the amount of practice driving completed during the learner stage of graduated driver licensing offers a step toward a performance-based licensing system.^13^ This approach would make progression to the on-road skills test and independent licensure contingent on the completion of the required number of practice driving hours on the basis of objective behavioral data. This may encourage more practice by preventing learner drivers from falsifying logbooks in states where they are required and creating an expectation that the required number of supervised practice hours is mandatory for progression through the licensing system. The feasibility of using this approach could be established in a single state, which would determine large-scale feasibility and user acceptance, including logistical aspects and understanding any implications for equity.
The findings of this study show that the use of an electronic logbook is a viable approach to measuring practice driving during the learner stage of licensure. Except during the first month of data collection, there was no difference in the self-reported amount of driving between those who used the paper and electronic logbook groups, suggesting that practice driving behavior is not impacted by the data collection method. Most study participants using the electronic logbook reported high levels of satisfaction and were willing to continue using the app beyond the study period.
The Johns Hopkins School of Public Health IRB approved the study (IRB#8283).
Declarations of none.
Johnathon P. Ehsani: Conceptualization, Investigation, Methodology, Software, Writing - original draft. Rebecca Weast: Conceptualization, Writing - review & editing. Theresa Chirles: Data curation, Investigation, Supervision, Writing - review & editing. Andrew Hellinger: Data curation, Supervision, Writing - review & editing. Gayane Yenokyan: Formal analysis, Writing - review & editing. Yifan Zhang: Data curation, Formal analysis, Writing - review & Editing.