reward a winner, but to accelerate the development
of cutting-edge technology to benefit cancer patients.
CAMELYON16 attracted 32 entries. Each entrant
received 270 digitized slides of lymph nodes removed
from patients who had undergone breast cancer surgery. Their task was to accurately identify the slides
that showed cancer by using computer algorithms,
mathematical recipes written in code to teach a
computer how to do a task. The winning entry,
a joint effort by researchers at Harvard Medical
School in Boston and the Massachusetts Institute
of Technology in Cambridge, used an artificial neural
network—a sort of computer brain—to tell the
difference between tumor images and nontumor
images. It had been trained to recognize cancer by
reviewing millions of such images.
After the winner had been crowned, the researchers gave the test data to 11 pathologists. The top five
digital entries in the contest correctly identified
metastatic growths at least as accurately as a pathologist who had as much time as needed to study the
slides. Under a time constraint—two hours to study
129 slides, a typical work rate—the pathologists’ diagnostic accuracy fell significantly short of the winning
entry’s. In many cases, the algorithms used pattern
recognition strategies to detect metastatic growths
that none of the pathologists found.
The group that sponsored the contest published
its research results in the Dec. 12, 2017, edition of
JAMA. “We were shocked to see that the algorithms
could do so well so quickly,” says lead author Babak
Ehteshami Bejnordi, who organized CAMELYON16
while completing his doctorate in machine learning at Radboud University. He’s now an engineer
at Qualcomm Research Netherlands, a technology
company in Amsterdam.
Artificial intelligence, or AI, is the broad term
used to describe efforts to get computers to function more like human brains and perform tasks
ordinarily done by people. Headlines about AI and
cancer often focus on efforts like IBM’s Watson
for Oncology initiative, which analyzes data from
thousands of patients to provide oncologists with
treatment guidance and recommendations. Nearly
seven years after its introduction, however, IBM’s
program is still in its infancy, and no peer-reviewed
journals have published studies showing that Watson
for Oncology benefits patients.
Currently, a more focused kind of AI is making
a difference in other areas of cancer research.
Machine learning, an AI approach that’s particularly good at pattern recognition, is already driving
the development of image-processing tools that can
help pathologists and radiologists learn more about
patients’ tumors faster than ever before.
The Signal and the Noise
Ehteshami Bejnordi says the test for CAMELYON16—
scanning lymph nodes for metastases—was selected
because it’s an important, routine and time-consuming
task for pathologists. When cancer spreads, it often
shows up first in nearby lymph nodes. Breast cancer
spreads to lymph nodes in the armpit early in metastasis, which is why they are often removed and
analyzed during mastectomy or lumpectomy.
Pathologists have to learn how to distinguish the
signal from the noise, that is, see patterns in cells of
a tissue sample that indicate the presence of cancer.
Radiologists, too, depend on recognizing patterns
when looking for tumors in high-resolution images
captured by CT scans, MRIs and PET scans. Such
images can reveal the location, size and shape of
a tumor, and that information influences treatment decisions.
However, there are limits to the abilities of patholo-
gists, says Gann. Pathologists zoom in and count
individual cancer cells within “hot spots” on a slide,
places they suspect they’ll find cancer. “But they can’t
possibly count all the tumor cells in the slide,” he says,
“and two pathologists who approach the same case
may not examine the same hot spot.”
“It’s really easy to miss the little things,” says
Kate Lillard, the chief scientific officer at Indica
Labs, based in Corrales, New Mexico. Indica is
developing software that enables pathologists
to harness AI approaches and more accurately
to learn how to
signal from the noise.