The scariest and most unrealistic briefing paper I’ve ever read is written from the perspective that NHS systems may be trivially upgraded to support the dreamware being proposed, as NHS systems are all modern, performant, homogeneous, well-administered and capable of delivering the authors’ goals with negligible “exemptions” or additional “administration”.
“Errors” they care about are failures to detect CSAM, not “people being in pain because doctors can’t share medical images amongst themselves”.
In case you missed it, the House of Lords and Parliament in general is being pitched a white paper [PDF link] which basically proposes “let’s put spyware in everyone’s phones to address child abuse, rather than actually go out and fix the situations in which children are abused”.
It’s a cloud-cuckooland document, but for British people perhaps the least realistic aspect of it is that it envisions the NHS as “tele-medicine” with the administrative, technical and flat-out hardware capacity to survive under the yoke that they propose:
Quote:
A model can therefore be trained to distinguish abusive or exploitative circumstances from lawful family, medical, educational, artistic or recreational contexts. Visual similarity alone may not be sufficient where the legal and safeguarding distinction depends on the activity, setting, pose, relationship between subjects or wider context.
And:
Not all missed detections have the same consequences either. Failing to detect clear and severe CSAM may present a different safeguarding risk from failing to classify a borderline, low-quality or technically ambiguous case. Performance should therefore be assessed not only by how many mistakes occur, but also by what kinds of mistakes occur, how those mistakes are distributed and what consequences they create in practice.
The seriousness of an error will also depend on the setting in which it occurs. A temporary false-positive warning in a low-risk environment is not equivalent to blocking legitimate medical or family content, while a missed detection during live capture or streaming may allow harmful material to be created, viewed, transmitted, saved or shared. The same statistical error rate may therefore have very different practical significance in different contexts. Whether a particular level or type of error is acceptable to users, corporations, and policy makers will also depend on the action that follows the model’s decision.
And:
Classifiers designed to detect CSAM are developed so that nudity in general is not automatically flagged on its own. This supports a more accurate distinction between potentially unlawful material and benign images that may contain nudity without constituting CSAM. Training and evaluation should include representative benign and borderline content so that the model learns not only what should be blocked, but also what should not be blocked. This is particularly important for reducing false positives in ordinary family, medical, educational, artistic, or age-ambiguous contexts.
And:
Regulation can assess whether to allowlist patient-to-provider telehealth applications and other low-risk applications to prevent on-device CSAM blocking from potentially interfering with legitimate virtual medical appointments. Any exemption should be narrowly defined, risk-assessed and capable of review so that it does not create an avoidable safeguarding gap.
And:
Exemptions apply to law enforcement and intelligence agencies for official investigative, forensic, training, and analytical work; licensed medical professionals in accredited institutions; poten- tially approved research institutions under data governance agreements; and child protection organisations operating under formal agency partnerships with oversight requirements.
And:
A regime that blocks defined categories of content on consumer devices necessarily blocks that same content on the devices of people whose lawful work requires access to it. The clearest examples are:
• Law enforcement officers engaged in the investigation of child sexual exploitation and abuse offences, who must be able to review seized material, examine devices forensically, and handle evidence on operational systems.
• Analysts at designated safety organisations, notably INTERPOL, the Internet Watch Foundation in the UK, the National Center for Missing & Exploited Children in the US, and similar organisations in other jurisdictions, whose functions include the assessment and categorisation of suspected illegal material.
• Forensic examiners in commercial and government laboratories supporting criminal proceedings.
• A narrow set of other roles—for example, certain medical, academic, or legal personnel whose work intersects with such material under specific authorisations.
[Again, PDF link if you want to read the document]
As we learned during lockdown, NHS systems are a heterogeneous bunch of information sharing solutions, stitched together with out of date and poorly administered window systems and the iron will of medical practitioners.
If anything remotely like this document is adopted, things will not go well. Things will in fact go very badly indeed.
Previously:
Leave a Reply