What is the appropriate recall rate of logistic regression in the field of disease screening?

There is no one-size-fits-all standard answer. Judging from the common practice of domestic public health and clinical screening, for the screening of malignant diseases with rapid progress and poor prognosis, the recall rate of the logistic regression model must be stable at least 90%. For the initial screening of people with common chronic diseases, it is generally stuck in the 80% to 85% range, which is enough to adapt to the current allocation of medical resources.

After all, the core requirement of our screening is not to miss a diagnosis. To put it bluntly, the recall rate is the proportion of people who are really sick and can be detected by the model. Every one percentage point lower means that a group of patients who could have been detected and intervened early are missed. In diseases such as tumors, missed diagnosis may be the difference between life and death.

But not everyone agrees with this standard. This issue has been debated for many years and there is no unified standard. In essence, the logic of settling accounts in different scenarios is completely different.

For example, grassroots public health personnel who are doing primary screening in areas with high incidence of gastric cancer in underdeveloped areas will probably feel that the recall rate of 85% is completely insufficient - the local people's follow-up compliance is less than 60%. Your model first missed 15%, and 40% of the remaining positive patients did not come for reexamination. In the end, very few patients were caught. In this scenario, everyone would rather lower the discrimination threshold and increase the recall rate to over 92%. Even if there are more than ten percentage points more false positives, at most it will be better to perform a gastroscopy on a few more healthy people than to miss one advanced gastric cancer. After all, the cost of the latter is a life.

However, practitioners in the screening center of the Third Hospital most likely do not think so. The amount of screening they receive every day is already saturated. For example, for the primary screening of radiomics with low-dose CT for lung cancer, if the recall rate of logistic regression is forced to 95%, the false positive rate can reach 35%. Originally, 1 Only 40 out of 000 screeners had real lung cancer. As a result, the model circled 390 people who needed further puncture. Not to mention that the puncture room number was not available, so many people who were fine were frightened and troubled by the positive report, which actually lowered everyone's trust in the screening. In this scenario, the recall rate will be stabilized at 90%, and the remaining 10% risk of missed diagnosis will be covered by a second review by the imaging doctor, and the overall efficiency will be higher.

I have previously participated in a pre-experiment on two types of cancer screening for rural women in a province. In order to control false positives, the recall rate of the logistic regression questionnaire preliminary screening model was stuck at 82%. As a result, three early-stage cervical cancers were missed during the small-scale verification. Later, the discrimination threshold was lowered. By two points, the recall rate increased to 89%, the false positive rate only increased by 6 percentage points, and the workload of subsequent colposcopy review only increased by less than 10%, which was completely within the capacity of grassroots maternal and child care. This threshold was used for the last four or five years, and the feedback from the grassroots level was good.

It’s also interesting to say that many algorithm engineers who are new to the industry always think about making the recall rate as high as possible. They forget that logistic regression itself is a linear model. The advantage is that it is highly explanatory. When explaining to ordinary people why you are judged to be high risk, they can list it one by one, "Your age, HPV infection history, and family history account for the high score." Compared with Those black-box deep learning models are much easier to implement, but precisely because of their limited fitting capabilities, if the recall rate is too high in complex indicator screening scenarios, if the recall rate exceeds 95%, there is a high probability that it is overfitting. If people in another area use it, it will collapse immediately. It is better to have a slightly lower recall rate, coupled with manual review, which is safer.