Accuracy is an interesting thing in medicine. There is an assumption, this unspoken hope, from those who seek medical care that the answer they get is correct. Certainty breeds confidence, and confidence (without reference to accuracy) breeds a measure of certainty. A diagnosis, confidently stated can lead to a feeling of reassurance, and that the answer/the diagnosis is accurate. Physicians are intensely trained, and have a high level of professionally acquired skill. However, medical practitioners are not perfect. The idea that medical purveyors are inacurate, and do not come up with the correct answer the first time (or the second or third time) propels the success of television dramas like House, M.D.
When thinking about curating a medical database of many user submitted images, the topic of accuracy arises. Can we trust a citizen, a user, a non-professional to contribute an image of a skin lesion? Will it be labeled with the correct diagnosis? In my mind this is an open question. As described previously, we like to share images - think about the number of pictures uploaded to Facebook daily. The number of encounters with medical professionals related to skin issues far outstrips the number of pictures in currently available databases (see prior posts). It is the belief of this project that people can be trusted to contribute to a scientific project in a meaningful (meaning accurate) way. As a first pass, the uploaded image will rely on the diagnostic accuracy of the medical professional who evaluates the rash or lesion. It is trust given to the user/image contributor that they are contributing what a medical professional has communicated.
The question that has been in my mind recently – How accurate are medical professionals when pronouncing a diagnosis regarding the skin?
I conducted a review in the published scientific literature and come across some information shared below which will shed some light on this question. This is by no means an exhaustive review. If you are interested in the full paper please email me, however they are easily accessible via Pubmed. Below you will find a listing of medical/scientific studies from the 1970s through to the present which look into this question. The year of the study publication is listed as well as the title of the medical paper and a brief description along with an accuracy rate in the text. In many cases accuracy is defined as the diagnosis given to the picture or patient after examination as compared to the correct diagnosis which is typically a biopsy-which serves as the final arbiter of correctness in skin diagnosis.
There is a range in the level of accuracy. It is not surprising that dermatologists, who have more experience with skin have higher accuracy rates. It is interesting to note that accuracy is not 100% for dermatologists all of the time.
With regard to a skin image database another questions arises: Are 10,000 images that are 99% accurate better or worse than 1,000,000,000 images that are 60% accurate?
At this point I think it is still an open question.