Fail factors

DIA Insurance ADI News

An unexpected outcome of the COVID-19 pandemic was a petition to the government, that was started by Ben Stanway, asking them to ‘allow learner drivers to be passed if their driving instructor feels they are safe to drive due to the failure of the government to enable learners to book a test’. 

Part of the statement by the government in response to this petition, which can be found on petition.parliament.uk, says:

“Although ADIs are well qualified and proficient in driving and instruction, they are not experienced assessors and this is evidenced by the current practical test pass rate of 47%.”

This is not the first time the finger has been pointed at driver training regarding the pass rate. Driver training is also frequently blamed for the disproportionate number of collisions that young drivers have. This article will focus on why the above quote regarding the pass rate is too simplistic. There are many factors that affect performance on the driving test, and I would argue that factors concerning the individual rather than their driving instructor account for most of the variance observed in the pass rates.

An important element of my university degrees in psychology was concerned with scientific methodology and proving that observed results on the dependent variable were not due to chance but could be attributed to the independent variable in our experiments. A correlation occurs when one variable influences another one. For example, there is a correlation between temperature and water causing a change in state going from a solid below 0°C to a liquid above 0°C and then eventually to a gas above 100°C. 

Multiple regression is a statistical analysis that considers the impact of several factors on a dependent variable. It is not possible for me to provide a multiple regression here explaining the amount of variance each independent variable has on the dependent variable (the pass rate) but I can consider the different factors and show how they would likely be a significant factor in a multiple regression carried out with the pass rate being the dependent variable in the analysis. 

Gender

To keep my approach scientific, I shall use pass rates published by the DVSA that are presented in Excel files on GOV.UK. I shall avoid using data collected during the COVID-19 pandemic and focus on the five financial years leading up to and including 2018/2019. The first factor to be considered will be the candidate’s gender (table 1).

Table 1 shows that the difference in the national pass rates between males and females is remarkably consistent. On average, across the five years considered, the pass rate for females is 3.64% lower than males. With the average number of tests conducted each financial year across this time period being 1,636,779 it is likely that this difference is statistically significant. Therefore, gender can be considered a factor that accounts for 3.6% of the variance in the practical pass test rate. 

It is likely that a significant factor accounting for this result is confidence with males being more confident than females, a finding that is not only found in driving but other domains as well, for example sports. 

When looking at the data more closely, it can be observed that the difference in pass rates between males and females varies across the test centres. For example, at the Wolverhampton test centre in 2018/19 the difference was 9.3% whereas in Westbury it was only 0.8%. Therefore, the next variable to be considered in this analysis is the centre where the candidate is taking their test.

Test centre

How the pass rates vary in different locations sometimes pops up in the media. In a BBC news report (22/04/2019) the pass rate was reported as ranging from 29.6% in Birmingham (Yardley) to 84.9% in Inveraray. There were only a few tests (53) at Inveraray so this could skew the results but when you look at Llandrindod Wells (with 464 tests) the percentage is still much higher than Birmingham with a pass rate of 74.1%. 

The table on the next page shows the difference in pass rates at 14 centres where there were at least 7,000 tests conducted in 2018/19 (table 2). In this table the centre with the most tests (Isleworth) has not got the lowest pass rate and the centre with the fewest tests has not got the highest rate (Portsmouth). Also, test centres in similar areas have different pass rates. Wood Green and West Wickham are both in Greater London but the pass rate at West Wickham was 7.5% higher in 2018/19.

These results mean that we need to consider other factors, assuming that the performance of both ADIs and examiners are consistent in all areas which for the purposes of this analysis I believe they are. The fourth column in table 2 shows the Index of Multiple Deprivation Rank (IoMDR) for the constituency the test centre is in. The IoMDR is based on 39 separate indicators that have been classified into seven domains: Income (22.5%), Employment (22.5%), Education (13.5%), Health (13.5%), Crime (9.3%), Barriers to housing and services (9.3%) and Living environment (9.3%). 

A higher score in the fourth column of table 2 indicates a constituency is relatively more deprived. It has been shown several times that there is a link between deprivation and people being injured on the roads, so it is safe to say that this is a broad indicator of road user behaviour and therefore has a bearing on how people view learning to drive and taking a test. 

I carried out a test to see if there was a correlation between the Pass Rate and Constituency IoMDR and found there was a correlation coefficient of -0.68 (which is statistically significant). This result shows a negative relationship between IoMDR and the pass rate, that is the higher the IoMDR ranking for a constituency the lower the pass rate is. 

A factor which would affect pass rates, that goes along with a higher IoMDR as they are usually in dense urban areas would be the nature of their roads. Driving around Wolverhampton where I do most of my teaching would be very different to driving in Ludlow in south Shropshire, for example.

Other factors

Going back to considering individual differences, age is a factor when people take a driving test with the pass rate declining as people get older. For example, the pass rate for 17-year-old candidates in 2018/19 was 55.8% with this declining steadily as people age (eg it is below 40% for people in their 30s). 

People who attend the test in an automatic car are more likely to fail with the pass rates for automatic tests over the five financial years being considered here to be 39.8% whereas the overall pass rate was 46.62% for the same time period. Given the numbers involved, a difference of 6.82% is likely to be statistically significant and we must consider the reasons that people choose to attend in an automatic as factors affecting the pass rate (eg having trouble with the controls or thinking you can pass quicker in an automatic as it is easier). 

Regarding the car that people attend the test in, the pass rate is slightly higher for those who attend in their own car. The most significant factor here is the extra experience gained through having practice in addition to lessons, but people may feel more comfortable driving the car they are in most often.

I keep records of how many lessons people have, whether they have practised, the attempt they pass their practical test on and the number of faults that are recorded. I can only comment accurately on those who have only had driving lessons with me as I cannot be sure of how many lessons they had before. This accounts for one independent variable (the ADI) but not for some others (eg the test centre, or the examiner conducting the test). 

People who have extra practice do significantly better with a first time pass rate of 77% compared to those who only have lessons whose pass rate is 50%. As expected, people who have practice have fewer lessons (an average of 45 hours compared to 62 hours) but they also have fewer driver faults recorded on their tests. The advantages of experience and extra confidence gained by having practice make it (extra practice) a significant factor when considering the practical pass rate, and if the difference of 27% I have experienced with my clients is reflected among my colleagues (which I strongly believe to be the case), it is probably the biggest factor accounting for variance in the pass rates.

Human error

It is well known that human error is the biggest contributory factor for collisions, and I am confident it is a significant factor accounting for the national average pass rate as well. It would be difficult to accurately obtain data relating to this variable, but I am sure many, if not all, of my colleagues have had experiences like the one I am about to give as an example. 

I was on a pre-test lesson with a candidate who had done everything right in her approach to learning to drive and I asked her to follow the road ahead at a roundabout near to the test centre, which she did in textbook fashion. Twenty minutes later I was sitting in the back of the car during her test, we were approaching the same roundabout in the same direction as on the pre-test lesson. The examiner asked her to follow the road ahead and I thought “this is no problem; she was here just now and drove it really well”. She then proceeded to take the right-hand lane instead of the left and then cut back across to take the second exit and failed her test. There are many examples I could relate, and I have heard clients say afterwards “I knew at the time I was doing something wrong, but I still did it”. It would be interesting to undertake some research into why people feel they make mistakes like the one outlined above while taking a driving test.

To summarise, I have tried to take a scientific approach to examining factors that affect the practical pass rate adversely. It is not an easy matter to accurately state the reasons the national pass over the five years focused on in this article was 46.62%. There are many factors involved and doing research in an applied setting is always going to be plagued by confounding variables. 

Having said this, I can say with some confidence that I have outlined six independent variables that adversely affect the pass rate: gender, age, the IoMDR of the constituency the test centre is located in, why people choose to learn to drive an automatic car, people not having practice in addition to driving lessons and human error. 

If the pass rate is due to ADIs being poor assessors, could we also say that the driving test regime is poor at identifying drivers who will be a risk in the future due to the higher pass rate in young drivers yet a disproportionate number of this group of drivers have collisions once they have passed the practical test? Of course not, many of the factors that affect the accuracy of the driving test in relation to the future behaviour of drivers are the same as I have been discussing in this article, along with others such as peer pressure, car sub-cultures and personality.

The post Fail factors appeared first on Driver Trainer.


Source: ADI News

Why not share this post?