There is a national aim to improve grading quality across the NHS Diabetic Eye Screening (DES) Programme to:
- reduce variability in screening outcomes between local services
- improve the programme for people with diabetes
PHE has done a lot of work around validating a method to quality assure the grading of images taken during DES appointments.
We have published new national guidance that describes a statistical method to compare grading outcomes between local screening providers. This method identifies providers with unusual grading outcomes compared to all the others.
These providers are identified as ‘atypical’ and might need to improve grading before implementing the UK National Screening Committee recommendation for extended 2-year intervals for people at low risk of developing retinopathy.
We intend to use this as a validated annual quality assurance tool. It will also form part of the quality measures for selecting local providers who will be our ‘pathfinders’ for the extended screening intervals pathway.
Pathfinder providers are those that will be the first to implement extended intervals and pave the way for all other providers to follow.
The data is taken from quarterly reporting submitted to PHE. We will use data submitted for quarter 4 2018 to 2019 (1 January 2019 to 31 March 2019) to calculate the atypically scoring for the 2018 to 2019 screening year.
The new grading quality report will be ready at the end of the second quarter of 2019 to 2020 (September 2019). This will be available to providers, commissioners and the screening quality assurance service (SQAS).
The full guidance can now be found on GOV.UK. All queries should be directed to the PHE Screening helpdesk.
Test and Training successfully quality assures national graders
The national Test and Training (TAT) system is an internet-based resource that local screening providers use to quality assure all grading staff by recording how good they are at detecting diabetic eye disease.
We have developed some new features to help graders and their managers monitor grading performance within the system.
R3 training sets
Graders have been asking us to include stable treated proliferative retinopathy (R3S) images in the monthly test sets.
To check an image for stable pathology you need to have a baseline image for comparison and this is not the current method for presenting images in TAT. Because of this, we have introduced an R3 training set which has both the previous and most current images so graders can compare images.
The new training resource allows graders to check:
- their understanding of stable R3S pathology and what it looks like
- what features they can expect to see when a previously stable treated retina reactivates with new R3A features
There are 10 images sets. After each set, the grader will be able to see their agreement with the guide grade. The training sets can be used as often as needed and do not expire.
There is no ‘pass mark’ and these R3 training sets do not contribute to the monthly TAT scores.
Accessing images from the guide grade level of agreement table
The ‘Level of Agreement’ table is a useful report which quickly shows graders their agreement with the guide grade. Graders use this to see where they have over or under graded in the test, and if they have any disagreements in the red boxes (R3A cases which would not have been referred).
It is now easy for graders to go to the image set where there is a disagreement by clicking on the new links in the table, as each cell is now ‘active’ (see image below).
Flagging graders on long-term leave
To help providers monitor grader absences and participation there is a feature to flag graders who are on long-term leave and unable to do the monthly test.
The programme manager can use the toggle on and off button and set date parameters which cover the period of leave. This feature (see screen shots below) is useful for reporting grader participation to commissioners and SQAS.
Using NHS email to register TAT accounts
Over the next 12 months we would like all users to use an NHS email address in their registered TAT account. This will make it easier for TAT providers to monitor applications and prevent duplication of accounts. All staff working for either a public or private provider can obtain an NHS email as part of their screening role.
We have confirmed that all staff can register for an nhs.net email account. If you do not have an NHS email you will need to speak to your IT department which can arrange this for you.
If you are a grader for more than one service in England you will need to ask the TAT provider to share your results between those services. You will need to select one provider as your primary account and share the results with other providers. This means you only use one login account to do the test once a month.
The same is true if you work as a grader for a screening programme outside England (such as Ireland), where the international TAT system operates. You should only do the monthly test set once, not once on each system.
Please use the new helpdesk email address for technical queries: drshelpdesk@drseqa.org.
The link is displayed on the left panel on the TAT website.
PHE Screening blogs
PHE Screening blogs provide up to date news from all NHS screening programmes. You can register to receive updates direct to your inbox, so there’s no need to keep checking for new blogs. If you have any questions about this blog article, or about population screening in England, please contact the PHE screening helpdesk.
4 comments
Comment by Irene Stratton posted on
So it's been modelled but not tested?
Thank you.
Comment by Andrew Anderson posted on
Dear Irene
Some programmes have been involved in an audit to validate the modelling. The data will not replace other grading quality measures. It is intended to offer services an additional report and a way of identifying grading outcomes which are unusual when compared to other services.
Kind regards
Comment by Irene Stratton posted on
Has this new methodology been trialled to see whether those who are identified as 'atypical' have problems with the grading? Or that those who are well within the limits do not? Has one programme been looked at?
I can't see that the methodology has been tested from the document that's available.
Comment by Andrew Anderson posted on
Hi Irene
Thank you for your comment.
The methodology was thoroughly assessed and developed by the diabetic eye screening programme, using appropriate statistical modelling. We hope to publish this in the next 12 months.
Kind regards