The world is full of stochasticity, and a little bit of it has spilled over the delivery date of our November digest as well (🤒🤧). But the wait is finally over, and November’s roller-coaster 3x10 is now here. In this edition, you’ll read 3-sentence summaries of papers on deep learning advances in both weather prediction and the detection of pancreatic lesions from images, detailed mechanistic description of how cancers hijack neurons and the immune system to their benefit, synthetic genomes, causal inference, new neurons in pregnancy, and more 🚀🧬.

Enjoy 3x10!

1. Weather prediction finally getting real (Lam et al., Science). One of the longest-standing arguments against the usefulness of computational modeling in general (and deep learning in particular) has now just crumbled: “we can’t even properly predict the weather” - thanks to DeepMind, now we can. The model, GraphCast, is implemented as a neural network architecture (and based on graph neural networks), can be ran on a desktop computer, and produces super fast (<1 minute) and super accurate weather predictions. The model is first trained using estimates of past global weather from 1979 to 2017 made by physical models, then it uses both the current state of the weather, as well as weather estimates of the past 6 hours, in order to predict the weather 6 hours ahead - and this estimation trick is used to successively push predictions further into the future.

   

2. How CAR-T cells can become unfriendly (Lareau et al., Nature). Anti-cancer chimeric antigen receptor (CAR) T cells are first extracted from cancer patients, artificially engineered to fight against cancer cells in a targeted and precise manner, and then infused back into the patients. Unexpectedly, a very rare population of these T cells (1 in 300 - 10,000 cells) can sometimes become “super-expressors” of the human herpesvirus 6, a virus which in some cases causes brain inflammation and can be very dangerous. This study offers a mechanistic justification for anecdotal medical reports, and will hopefully inform the future design and improvement of CAR-T therapies, such that this newly identified potential complication is circumvented.

   

3. Epigenetic regulation during cancer transitions across 11 tumour types (Terekhanova et al., Nature). It has become increasingly clear over the past few decades that not all alterations driving cancers are genetic, and that the epigenome does influence cancer initiation and progression in essential ways. This new study has generated single-cell chromatin accessibility data for 1,000,000 cells from 200 tumor samples of 11 different cancer types, using single-nucleus assay for transposase-accessible chromatin (a large part of which was also paired with scRNAseq expression profiling). Valuable dataset for the community to mine further and, as expected, some regulatory regions called as drivers are unique to a single cancer type, whereas others are shared among types.

Subscribe now

4. Living and replicating yeast genome with over 50% synthetic DNA (Zhao et al., Cell; Schindler et. al, Cell; Taghon et al., Cell Genomics). The genome of Saccharomyces cerevisiae (aka yeast) has 16 chromosomes: in this new artificially-designed strain, 6.5 of them were synthesized in the lab, and an additional chromosome was created by combining various edited bits and pieces. Doing this is not easy, and the Sc2.0 consortium (which consists of labs from all around the world) has been working towards the goal of creating a fully artificially engineered yeast for the past 15 years. A main hurdle to overcome was eliminating potential sources of instability in the yeast genome, by removing repetitive noncoding DNA regions, as well as relocating regions that code for transfer RNAs from their original positions spread across the genome into a new specialized synthetic chromosome (wow, this sounds totally crazy).

   

5. Pregnancy transiently remodels olfactory function in anticipation of motherhood needs (Chaker et al., Science). Lots of things happen during pregnancy, which remains one of life’s deepest mysteries, but the birth of new neurons isn’t necessarily something most people would expect. This cool study shows how during mouse pregnancy, multiple neural stem cell pools are activated in mothers and generate specific olfactory bulb interneurons that function around birth to modulate aspects of maternal care, including own-pup recognition, and then disappear as pups mature. This evidence contributes to understanding how the heterogeneity in neural stem cells might provide a substrate for adaptive brain plasticity in response to different physiological states (nice Twitter/X thread).

   

6. Deep learning models outperform radiologists in detecting pancreatic lesions in multiple medical centers (Cao et al., Nature Medicine). Pancreatic cancer is a very aggressive disease with a poor prognosis, usually diagnosed too late and in urgent need for effective early detection methods. This study describes PANDA, an AI imaging-based screening tool, trained on more than 3,200 image sets (70% of which from patients with a pancreatic lesion) and validated in a multi-center cohort of 6,200 patients, as well as a second cohort with 20,000 patients from the same institution. The algorithm achieved a sensitivity of 92.9% and a specificity of 99.9% for the identification of pancreatic lesions, with an area under the curve (AUC) of 0.986–0.996, and identified pancreatic adenocarcinomas better than radiologists.

   

7. Brain tumors hijack plasticity mechanisms to enhance cancer growth and ensure survival (Taylor et al., Nature). This important paper carefully characterizes the way in which brain tumors use a previously described neural plasticity mechanism which boosts neuronal activity (TRKB & BDNF related signaling) to malignant purposes. Specifically, BDNF supports the proliferation of malignant brain cells in mice transplanted with human cells and in cancer cells grown in vitro. Knowing that the survival and malignancy of brain tumor cells are boosted by their ability to hijack neural mechanisms that increase synaptic strength is not a good news - but hopefully therapies targeting these neural interactions can be employed to slow down brain cancer growth.

   

   Subscribe now

8. Another bad news: macrophages can also support tumor cells, this time in pancreatic cancers (Caronni et al., Nature): a specific population of tumor-associated macrophages (TAMs) which express interleukin-1β (IL-1β) is responsible for sustaining an inflammatory loop together with tumor cells. Following lots of experiments and genomic single cell and spatial analyses, this reprogramming process was found to happen early in tumor development and to be distinguished by lasting transcriptomic changes in tumor cells, as well as specific spatial patterns, further leading (or at least being associated with) poorer patient outcome. On a potential future therapeutic ray of hope, blocking the activity of interleukin-1β helped to control the growth of pancreatic tumor cells (in mice).

   

9. Extensive single cell resource on immunaging and how the immune system changes with age (Terekhova et al., Immunity): profiling 2 million single cells with single cell RNA sequencing and TCR/BCR sequencing of 317 samples from 166 healthy individuals aged 25–85 years old. 12 out of the 55 identified subpopulations change with age: type 2 memory CD4⁺ and CD8⁺ T cells increase with age and can produce more IL-4, NKG2C⁺GZMB^–XCL1⁺ defines a unique memory CD8⁺ T cell subset that decreases with age and HLA-DR⁺ CD4⁺ memory T cells and GZMK⁺ CD8⁺ T cells accumulate with age. A healthy and potent immune system is absolutely essential for healthy aging, so understanding the natural changes that immune cells undergo with age is incredibly important for increasing healthspan and lifespan.

   

10. Finally, CINEMA-OT, a very nice new computational framework based on causal inference for single cell perturbation analysis (Dong et al, Nature Medicine) , has been used to investigate the cytokine language that activates immune response, revealing how different immune cells use combinations of cytokines to communicate to each other. Certain cytokines combine synergistically, and this combination of signals with novel outcomes acts as a language which directs the actions of the immune cells. The algorithmic innovation comes from computationally matching the states of different cells between control and perturbed populations, in order to infer the dynamic changes between the two conditions.

Welcome to October’s 3x10, our monthly roller-coaster journey through October’s coolest life science papers 🚀🧬. In this edition, you’ll read 3-sentence summaries of papers on evolution, single cell methodologies, genetic screens, and more. At the end of the newsletter, you’ll get an interesting video as a Halloween bonus 👻.

Enjoy 3x10!

1. Assembly theory (Sharma et al., Nature). The most (in)famous paper I read this month proposes a new framework (assembly theory, that is) to explain basically everything… or, more specifically, “to unify descriptions of evolutionary selection across physics and biology”. This paper is not an easy read for anybody (in particular evolutionary biologists), but, to its merit, it sparked scientific discussions by being different than what is expected for a scientific paper describing evolution. The abstract is particularly philosophical (and arguably tricky at times), and states that assembly theory “discloses a new aspect of physics emerging at the chemical scale, whereby history and causal contingency influence what exists”… food for thought, I guess.

   

2. Efficient and scalable compressed Perturb-seq (Yao et al., Nature Biotechnology). Perturb-seq is a technique that combines pooled genetic screening via CRISPR with single-cell RNA sequencing to assess the separate impacts of different genetic perturbations on a cell's gene expression profile in a single experiment for a cellular population (aka many CRISPR perturbations + scRNAseq readout, for a population of cells). Using an experimental and computational framework inspired by compressed sensing applied to genetic screens, this study manages to substantially reduce the number of measurements needed to run Perturb-seq, leading to an impressive reduction in experimental cost (and with only small changes to the original protocol). Compressed Perturb-seq is a wonderful example of innovative theory applied to wet-lab experiments with direct economic value, which will hopefully allow increased adoption of Perturb-seq screening.

   

3. Population-level integration of single-cell datasets (De Donno et al., Nature Methods): More and more single cell data is being added to single cell atlases, yet it is not clear how to optimally combine existing datasets to preserve biological signal and remove technical artifacts, or how to interact with collections of cells in the context of new data. scPoli is a computational framework for single-cell population level integration of scRNAseq & scATACseq datasets; an open-world learner that incorporates generative models to learn sample and cell representations for data integration, label transfer and reference mapping. The tool (available as an integral part of scArches) is able to rigorously complete a bunch of useful tasks when dealing with atlases: build references from semi-labeled datasets, compare donors, identify novel cell types, extend a reference with new cell types, and even integrate across species.

   

   Thanks for reading Life, explained! Subscribe for free to receive summaries of the 10 coolest life science papers in November 2023!

4. EVEscape (Thadani et al., Nature). A computational framework that combines fitness predictions from a deep learning model of historical sequences with biophysical and structural information, with the aim of predicting future viral evolution. EVEscape quantifies the viral escape potential of mutations and, by doing so, it manages to accurately predict nearly all known escape variants of SARS-CoV-2 from pre-pandemic sequence data alone. Moving forward, EVEscape can predict probable future SARS-CoV-2 mutations to forecast emerging strains as a tool to inform continuous and rational mRNA vaccine design, and to estimate the pandemic risk for previously unknown viruses.

   

5. Brain Cell Census (Science). In an impressive scientific effort, 21 papers published simultaneously shine light on the cellular composition and functionality of the most complex, magical and fascinating neural networks ever designed: our own brains. It comes as no surprise that these papers report an immense complexity of the prenatal, postnatal and adult human brain, consisting of thousands of cell types and states, with specific functionality. The papers are a valuable resource for the community and an important step forward in understanding living organisms.

   

6. Mosaic copy number alterations in normal tissues (Gao et al., Nature Genetics). This paper adds to the literature unravelling the evolution of somatic variations in normal tissues, and its relation to aging and diseases such as cancer. By inferring mosaic copy number alterations (mCAs) from the GTEX (Genotype-Tissue Expression) database of bulk gene expression profiles on normal tissues, the paper concludes that clonal expansions of mCAs are more widespread in normal tissues than previously thought: mCAs were found in more than 20 tissues, a quarter of the subjects harbored an mCA somewhere in the body, and the prevalence of clonal mCAs increased with age, from almost inexistent in people under 30 to 30% in people above 50. The prevalence and patterns of mCAs across tissues suggest tissue-specific mutagenic exposure and selection pressures, mirroring previous findings on the evolution of somatic point mutations in normal tissues.

   

   P.S.: paper #5 covered in September’s 3x10 edition also discusses the evolution of mosaic copy number alterations in healthy tissues.

7. The mother’s microbiome helps in the formation of the placenta (Pronovost et al., Science Advances). The authors of this paper treated pregnant mice with antibiotics that killed part of their gut microbiome and found that the placentas of antibiotic-treated mothers were smaller than those of mother mice with normal microbiomes. In addition, they had less-developed blood vessels needed to transport nutrients, oxygen and immune molecules from the mother to the fetus. This means that the gut microbiome plays an essential role in the formation of the placenta (in mice at least), further strengthening the link between mother’s microbiome and fetal development.

   

   Subscribe now

8. Identification of cell states altered in disease (Dann et al., Nature Genetics): This paper investigates the question of how to optimally utilize existing single cell atlases (especially data from healthy individuals) to identify cell states altered in disease, and whether matched case-control data is needed for this task. It compares the same workflow for disease-state identification using as healthy reference either i) an atlas dataset (easier to get, but less specific); ii) a control dataset (expensive and smaller, but very specific); iii) a combination of both of the above, where the atlas is used for latent space learning, and the control as reference for differential analysis. They find that the combined approach of using a reference atlas for latent space learning followed by differential analysis against matched controls consistently increases precision and leads to improved identification of disease-associated cells, especially with multiple perturbed cell types.

   

9. Resident regulatory T cells reflect the immune history of individual lymph nodes (Kaminski et al., Science Immunology). Regulatory T cells (Treg) are present in lymphoid and non-lymphoid tissues where they restrict immune activation, prevent autoimmunity and regulate inflammation: while Treg in non-lymphoid tissues are resident, those in lymph nodes (LNs) are considered to recirculate. In a beautiful series of experiments, this paper uses in situ labeling with a stabilized photoconvertible protein to uncover turnover rates of Treg in LNs in vivo. It finds that, similar to conventional T cells, Treg can form resident memory-like populations in LNs after adaptive immune responses, which might provide new therapeutic opportunities for the treatment of local chronic inflammatory conditions.

   

10. The cancer-immunity cycle collection (Immunity). I don’t usually do Reviews in this newsletter, but this 7-article collection deserves an exception. The cancer-immunity cycle is a framework to understand the series of events that generate anti-cancer immune responses, introduced in 2013 by Chen & Mellman. Now, 10 years later, these review articles discuss progress and advances in the field, while highlighting key challenges: a telling example of our evolving understanding is the complex role of the tumor microenvironment in facilitating, not just suppressing, the anti-cancer response.

    

You made it till the end, that’s impressive. Here’s the bonus I promised.

That was 3x10 for October 2023. It you have suggestions of cool papers to include in November’s 3x10, please reach out either via Substack (Chat or Notes) or Twitter.

Today marks an important day. Life, explained expands with a monthly newsletter called 3x10 (3 times 10) and coming out on the 30th of every month (sorry, February)! In each edition, I will write 3-sentence summaries on the 10 most interesting life science papers that came out that month (subjectively interpretable). 3x10 has a strong bias for innovative approaches in machine learning, cancer research, and genomics.

And now… full speed ahead on a roller-coaster journey through September’s coolest life science papers 🚀🧬!

Enjoy 3x10!

1. AlphaMissense (Cheng et al., Science): Beyond too much doubt, the star of this month, AlphaMissense from Google’s DeepMind is a deep learning model for predicting the clinical effect (harmful or not) of missense variants (DNA point mutations that change the amino acid sequence of a protein). Combining AlphaFold pre-training with fine-tuning on databases of population frequency of DNA variants in the form of multiple sequence alignments, this model achieves new state-of-the-art performance and is able to reliably classify whether 89% of ALL human missense variants are likely benign or pathogenic. Even though this model is bigger and better than its predecessors (e.g. EVE, ESM), how exactly to employ it in clinical settings remains to be assessed: how reliable and, most importantly, how actionable are AlphaMissense’s variant effect predictions?

   

2. Spatial predictors of immunotherapy response in triple-negative breast cancer (Wang et al., Nature): Despite being humanity’s most promising weapon against cancer, immunotherapy unfortunately only works for some patients, for reasons which remain unclear and non-actionable. This beautiful study analyzes mass cytometry spatial signal of 43 proteins in tumor tissues from 280 triple negative breast cancer patients (the most aggressive breast cancer subtype) enrolled in a clinical trial comparing chemotherapy to chemotherapy plus immunotherapy. It finds that the spatial structure and composition of the tumor and its immune microenvironment both at the start of treatment, as well as early on-treatment, are predictive of patient response to immunotherapy.

   

   Thanks for reading Life, explained! Subscribe for free to receive summaries of the 10 coolest life science papers in October 2023!

3. Early-Stage Breast Cancer Detection in Breast Milk (Saura et al., Cancer Discovery). This short paper is just unbelievable: cell-free tumor DNA shed by breast tumors is detectable via Next Generation Sequencing in breast milk before being detectable in blood plasma, and up to 18 months before diagnosis! You might be rightfully curious how can something like this be rigorously proven: researchers enrolled women with breast cancer diagnosed during either pregnancy or postpartum, and two of the patients had frozen milk samples from either a previous pregnancy (for the patient diagnosed during pregnancy) or as part of surveillance (for the patient with advanced maternal age), which were retrospectively tested (and found positive!) for tumor DNA. Pregnancy and postpartum-related breast cancers are horrific, both psychologically and clinically, and I wholeheartedly hope that such studies will help gather the knowledge necessary to cure this terrible disease once and for all.

   

4. A vertebral skeletal stem cell lineage driving metastasis (Sun et al., Nature). Some cancers (e.g. breast or lung), preferentially metastasize to the spine more than to other bones, for yet unknown reasons (an old theory proposed that coughing pushes malignant cells backward and directs them towards the spine, where they can seed - but nobody proved this). The structure and composition of the spine are unique to vertebrates and result from stem cells different than the ones producing the other bones, named “vertebral skeletal stem cells” (proved in this paper by inserting these cells into the muscles of mice and finding that their progenies resemble spinal structures). By deleting the gene coding for a specific protein secreted by the vertebral stem cells, cancer cells in mice stop spreading to the spine to a large extent (in mice at least), which causally and strongly links this novel stem cell population to the process of tumor metastasis, contributing to our understanding of one of the major killers worldwide.

   

5. Mutation rates and fitness consequences of mosaic chromosomal alterations in blood (Watson et al., Nature Genetics). Genomic mosaic copy number alterations are common in tumors and have long been thought to be specific to cancer, but recent research has shown that they exist in healthy tissues as well (sometimes decades before cancer diagnosis), proposing them as a molecular clock to map the continuous temporal development from normal tissues into malignant tumors. Using blood samples from 500,000 people from the UK Biobank, this beautiful population genetics paper disentangles quantitatively the temporal evolution of copy number alterations in the blood of healthy individuals, by quantifying the rates of mutation (i.e., change), selection (i.e., reproductive advantage) and genetic drift (i.e., chance). Through statistical modeling of the relative sizes of the blood clones carrying such alterations in relation to biological age, the authors find that mutations happen relatively rarely, however, their growth advantage can be substantial.

   

6. Learning single-cell perturbation responses using neural optimal transport (Bunne et al., Nature Methods). The theoretical framework CellOT aims to understand and predict responses of heterogeneous single cells to chemical, genetic, or mechanical perturbations from unpaired datasets, by learning the maps connecting post-perturbation cells to their pre-perturbation states. The underlying model is an ingenious combination of optimal transport theory and deep learning inference and is applicable to e.g. predict the drug responses of unseen cancer patients from single cell RNA sequencing tumor data. Still, challenges remain for cases in which either patient variability or drug-induced changes are very large - hopefully such challenges will be overcome with more and more data.

   

   Thanks for reading Life, explained! Subscribe for free to receive summaries of the 10 coolest life science papers in October 2023!

7. Mismatch repair deficiency is not sufficient to elicit tumor immunogenicity (Westcott et al., Nature Genetics). Back to understanding when and why immunotherapy fails, especially for patients who exhibit higher DNA mutational burden (more mutations), usually associated with improved immunotherapy responses (why? because the immune system finds more enemies - mutated neoantigens - to train and fight against). This paper proposes and analyzes two mouse models of lung and colon cancers with DNA mismatch repair deficiency (leading to a much higher mutational load) and finds - surprisingly! - that the resulting tumors do neither facilitate T cell infiltration nor do they show a strong immunotherapy response. This happens as a consequence of substantial mutational heterogeneity, especially at the subclonal (lower frequency) level, which blunts immunotherapy response and ultimately allows immune evasion.

   

8. A foundation model for generalizable disease detection from retinal images (Zhou et al., Nature). RETFound is a ChatGPT-like large language model that extracts incredibly relevant health information from 1.6 million unlabeled images of the human retinas alone. The pre-trained model is then fine-tuned on many downstream clinical tasks with labeled data for detecting various disease conditions (eye disease, Parkinson’s, heart conditions) and shows impressive performance across different ages and ethnicities. Retinal imaging is an even more valuable medical instrument than previously thought, and models built on this data could one day assist and guide medical professionals toward better diagnosis and treatment decisions.

   

9. A bioelectrical phase transition patterns the first vertebrate heartbeats (Jia et al., Nature). Many people (especially parents!) know that the heart is one of the first organs to function (a baby embryo’s heart starts beating at only 22 days old), but how exactly the absolute first heartbeat arises in a previously silent tissue remains a fascinating mystery. This paper did sophisticated live imaging on zebrafish embryos and managed to capture fascinating videos of hearts blinking for the first time. Large-amplitude Ca++ spikes (responsible for the beats) seem to emerge in all cells at once in a concerted tissue-level and phase-transition-like manner, driven however by a quasi-random origin “leader” cell that first develops spontaneous activity, further “inspiring” neighboring cells to gain synchronized frequencies (how amazing is that).

   Subscribe now

10. A wearable aptamer nanobiosensor for non-invasive female hormone monitoring (Ye et al., Nature Biotechnology). Hormones are notorious for fluctuating wildly even throughout the day, and blood tests only offer invasive and static snapshots into hormonal activity, limiting our understanding of hormones’ dynamic impact on health and well-being. This paper introduces a novel wearable device that monitors estradiol (estrogen) non-invasively from sweat: a bendable wrap that sits on the finger, just like a ring, and sends data wirelessly to a smartphone. The measurements of this device are validated in human participans, and backed up by a direct correlation between sweat and serum estradiol levels, as well as clear cyclical fluctuation trends in sweat estradiol levels during the menstrual cycle.

    

    ---

    That was 3x10 for September 2023. It you have suggestions of cool papers to include in October’s 3x10, please reach out either via Substack (Chat or Notes) or Twitter.

Today’s post is about the wonders of deep learning.

Instead of starting off with the promise though, let’s first address the skepticism: some researchers remain skeptical about the applications of deep learning in life sciences. One of the main reasons for their skepticism is the hype surrounding deep learning.

Hype can indeed be a non-optimal ingredient in the scientific mix, as it sometimes comes at the cost of:

1. decreased rigor (people rush to use trendy techniques as these can lead to fast publications)

2. shifting of focus from questions with high future discovery potential to questions with seemingly immediate returns

In this post however, we do not care about the philosophical hype surrounding deep learning. What we are discussing instead is a scenario (a newly published study) in which deep learning is providing a real solution to a real problem.

Subscribe now

1. Context & Significance

Pancreatic cancer is a terrible disease, usually diagnosed very late. Despite it being a rare cancer type (current incidence estimate is 13 yearly new cases per every 100,000 people in the US), the survival rate from pancreatic cancer is very low (Figure 1).

This is particularly evident when compared with other cancer types, among which pancreatic cancer is an unlucky outlier. Figure 2 illustrates this point quite well (even though the data refers to England only, it stops in 2015, and it lumps all pancreatic cancer stages together).

But there’s an important catch: some of the patients who are diagnosed very early can be cured. Not all, but some. Most pancreatic cancer patients are however diagnosed late, with locally advanced or metastatic disease (Figure 3). Important to keep into account though that longer survival times when diagnosed early can also be caused by the lead time bias (it’s all so complicated).

Now we are ready to ask one of the toughest questions out there: given an early tumor, can we tell whether it is curable? The answer is unfortunately a loud and clear no.

Here, I want to take a small detour and reflect on the philosophy of early detection (I also discuss this in a previous post: Why we can't yet cure cancer). Early detection means finding tumors early on. Or, better said, earlier than the time by which they are symptomatic. Such early-identified lesions represent a novel class of tumors: some that we hadn’t studied so far.

How should we approach these tumors? Should we treat them in the same way as we treat tumors currently: surgery, radiation, chemotherapy, targeted therapy, immunotherapy? This sounds like a very reasonable thing to try. But is this the only way, or is it the right way?

You might wonder: why does this matter?

To understand this, we can use a metaphor. Let’s think of tumors as animals (snails, turtles, rabbits, birds) living in a space which is surrounded by a fence. The fence is early detection (Figure 4). Snails and turtles are very slow, and, in our metaphor, they correspond to cancers that either never progress (snails), or progress slowly enough that their evolution doesn’t pose a major health risk (turtles). For snails and turtles, the fence is not necessary. Some other cancers (rabbits) might indeed progress both slow and fast enough, such that their detection is relevant, and treating them early on does improve survival. In our metaphor, the fence will actively keep rabbits from escaping. The worse class of tumors are the birds: tumors that grow so fast that, no matter the strength of the measures (the height of the fence), they will be lethal.

But, as Figure 3 shows, what we do know for sure is that: in order to stand the chance of being cured, it is essential to catch the disease early. Welcome to the concept of early detection (Figure 5).

Now, for catching the disease early, we need some form of screening. But whom to screen? You might think… well, everybody. Unfortunately, such an approach is non-optimal for (at least) these 2 main reasons:

1. Cost and insurance coverage arguments

2. For a disease with such a low incidence, the net benefit of screening can be greatly diminished by the false positives of finding the disease when the disease is not there (remember Bayes’ rule)

In other words, we need good ways to prioritize whom to screen. This is exactly the problem that this paper is addressing:

Given a large population (and their clinical records), and given the willingness to screen some people, how do we choose which people to screen such as to identify as many pancreatic cancers cases as possible in a given time interval?

2. Datasets

To understand how this method works, we need to first understand the data it is based on: clinical records. How does this data type look like?

Every time a person goes to the doctor, and they receive a diagnosis, the respective diagnosis is added to their clinical record. These diagnoses follow sequentially over time starting at the beginning of the recording history, one after the other. Across tens of years, they represent the temporal medical history of the patient. The assumption underlying this paper is that, across a large population, the set of temporal medical events trajectories is informative for future diagnoses.

But which exactly is the rule of information flow here? Well, this is something that we don’t know. But what we do know is that this rule is complex.

Too complex.

Put differently, complex enough that effective rule-based screening algorithms based on past and recent diagnoses do not exist.

This is an ideal scenario with lots of data for deep learning to mine. For this reason, several deep learning models have been designed and used successfully to assess and predict the risk of various health outcomes from large clinical records datasets. What this model brings to the table is explicitly using the longitudinal nature of clinical records, i.e., the temporal sequence of codes. This sequentiality might be particularly informative for predictive risk assessment in incremental time intervals.

Let’s discuss in more detail about how this is achieved.

Subscribe now

The paper analyzes 2 large datasets of disease codes:

1. 6.2 million longitudinal patient records in Denmark (24,000 pancreatic cancer cases), collected between 1977 and 2018 (Danish Public Health Registry, DNPR)

2. 3 million longitudinal patient records (3,900 pancreatic cancers) in the United States, collected between 1999 and 2020 (US Veteran Affairs, US-VA)

These two datasets are inherently quite different, which makes for an interesting comparison (Figure 6):

1. the Danish dataset is general (representative of the population of an entire country), while the US Veterans dataset is specific (representative of a selected sub-population with unique characteristics)

2. medical practice, billing and reporting are very different in Denmark vs. the US: the health records in the US Veterans dataset have shorter, but much denser disease histories than the Danish ones (strikingly, there’s a median of 188 records per patient in the US-VA dataset versus only 22 records per patient in DNPR)

3. the prevalence of known risk factors for pancreatic cancer is another distinguishing feature of the two datasets. For example, type 2 diabetes, one of the well-known risk factors for a pancreatic cancer diagnosis, occurs in 3 times more patients in the US-VA dataset than in DNPR (Figure 7)

3. Deep learning model

The framework described below is general, and, in this paper, it is applied for training, prediction and performance evaluation on both the Danish and the US Veterans datasets. It can also work in principle for any other clinical records dataset, provided that the disease codes used are relevant and informative.

3.1 Learning

Training is done on 80% of the data (training set). A development set (10%) is used to examine the performance for different hyperparameter settings and for model selection. Lastly, a fully withheld test set (10%) is used to evaluate the performance of the selected models (Figure 8, left).

The training data contributed by each patient consists of the temporal sequence of diagnostic codes for that patient, which we call the clinical diagnosis trajectory. This trajectory includes both disease codes corresponding to diagnoses, as well as timestamps indicating when these diagnoses were received.

Inspired by Natural Language Processing (NLP) training frameworks, the model is trained with augmented data for each patient, by considering all partial trajectories (of length at least 5) of disease codes. Therefore, multiple trajectories enter the training dataset for each patient, and each trajectory is incrementally larger than the previous one with the next disease code, in a temporal sequence (Figure 8, middle).

For cancer patients, only trajectories that end before a cancer diagnosis is received are included. For patients without a pancreatic cancer diagnosis, only trajectories ending earlier than 2 years before the end of disease records are included, in order to avoid potential undiagnosed future cancers that would bias the accuracy of the prediction labels.

Now, the timepoint at which clinical records (or available disease history) end is called the time of assessment. The metric to be optimized by the model is the probability of being diagnosed with pancreatic cancer at different time intervals after the time of assessment (3 months, 6 months, 12, 36, 60 months; Figure 9). In other words: what is the chance of a pancreatic cancer diagnosis in the next 3 months after the available disease history ends? How about in the next 6? How about 12, 36, or 60?

3.2 Architecture

The model consists of 3 stages (Figure 10):

1. embedding of the categorical disease features (level-3 ICD codes) onto a continuous low-dimensional latent space.

2. encoding the temporal sequences of diagnostic codes into feature vectors using either sequential layers (GRUs - gated recurrent units), attention layers (Transformers) or pooling layers (MLP - multilayer perceptron and a control bag-of-words model, which ignores the time and order of disease events). The encoding layer also includes age.

3. predicting time-dependent cancer risk, in the form of an estimated probability of pancreatic cancer diagnosis, following a given disease history. The parameters of the model are optimized by minimizing the prediction error: the difference between estimated risk and real outcome, encoded as a step function (0 before the diagnosis was received, and 1 afterwards), via a cross-entropy loss function. The loss function is averaged over the 5 time-intervals considered for prediction (Figure 10).

Further, some of the patients are prioritized as high risk, by using a threshold on the estimated risk. These are the patients which are proposed for surveillance and additional follow-up investigations (Figure 11).

3.3 Evaluation

Model evaluation is done separately for each time prediction interval. The following example given in the paper is quite intuitive as to what this means in practice:

“Consider the prediction score for a particular trajectory at the end of the 3-year prediction interval.

If the score is above the threshold, one has a correct positive prediction if cancer has occurred at any time within 3 years and a false-positive prediction if cancer has not occurred within 3 years.

If the score is below the threshold, one has a false-negative prediction if cancer has occurred at any time within 3 years and a true-negative prediction if cancer has not occurred within 3 years.”

The 4 tested modeling architectures (GRU, Transformers, MLP, bag-of-words) were assessed by 2 methods:

1. AUROC: the area under the receiver operator characteristic curve

2. Relative risk (RR) curve, quantifying by which factor a prediction method does better than a random pick based on the population disease incidence alone

For a realistic clinical implementation, the paper proposes advancing to a screening program the top 1,000 patients predicted to be at highest risk of developing pancreatic cancer in a given time interval.

Subscribe now

4. Model performance

4.1 Danish data

Among the 4 modeling architectures tested, Transformers showed the best performance, with a particularly impressive RR ratio (Figure 12, Danish data). It is nevertheless interesting how the AUROC performance of the various architectures is not that much different, including the control bag-of-words approach, which ignores the sequentiality of the disease codes.

For assessing model performance more thoroughly, the paper sets up multiple in silico relevant experiments, such as:

1. excluding from training the disease history events occurring in the period right before a pancreatic cancer diagnosis (3, 6 or 12 months before). There are two reasons why such a sensitivity analysis is relevant. First, these disease codes (unlike the others) may indirectly, but strongly, already cover the pancreatic cancer label, which may bias the predictions. What’s more, these diagnoses can lead physicians to suspect pancreatic cancer, without the need to rely on black-box algorithms. Outcome: as expected, the model without any exclusion performs much better. This means that the medical events happening closer to a pancreatic cancer diagnosis are highly relevant in predicting the disease.

2. taking into account only risk factors (disease codes) already known to increase pancreatic cancer risk, as opposed to the entire medical history. Outcome: the model trained on all disease codes performs much better (Figure 13). This result is one of the key points of the paper, and one of the strongest arguments in favor of the deep learning approach employed: using only the diagnoses that we currently think are informative is not enough, and additional information is extracted from the stuff we would not have normally considered relevant.

3. assessing how the prediction performance changes with different time prediction intervals. Outcome: as expected, the shorter the time prediction interval after the end of disease history, the better the performance of the model. This is because longer time intervals imply larger gaps in potentially informative medical information.

4.2 Cross-application

The impressive prediction performance discussed above comes from a model trained on a very large corpus of well-curated clinical data (remember, the Danish dataset had 6.2 million patient records, see Figure 6). Putting together such a high amount of high-quality data is prohibitive for many healthcare settings (at least nowadays). Therefore, a very relevant aspect to investigate is how applicable is this pre-trained model in a different healthcare setting?

Luckily, we can address part of this important question here, by applying the Danish-trained model on the US Veterans data, and the US Veterans-trained model on the Danish data! As discussed above in Section 2 (Figures 6 and 7), the two datasets are inherently quite different in several conceptual ways, therefore our prior expectation could be that cross-applying them would lead to substantial decreases in performance.

Let’s see whether this is true.

The Danish-trained model was cross-applied without any changes to its architecture on the US Veterans dataset (after only mapping the codes from one healthcare system to the other). As expected, its performance dropped substantially (Figure 14 a vs. b).

But how does the same model look like when trained and applied on the US-Veterans dataset alone? Indeed, prediction performance is again high (Figure 14c, compare with panel b), confirming that, when trained on a dataset with the exact same structure as the one used for prediction, the model does generalize well.

To drive home the point of a realistically-implementable clinical surveillance program, the paper identifies the top 1,000 patients (per every 1 million people) at highest risk of developing pancreatic cancer in a 12 months’ time prediction interval, with 3 months data exclusion (Figure 14d). The relative risk ratio guarantee is impressive: the patients proposed for follow-up are at least 59 times (Danish data) and 33 times (US-VA data) more likely to be diagnosed with pancreatic cancer in the following year than a totally non-informative prediction (disease incidence alone).

5. Feature interpretability

Lastly, the paper also addresses some aspects of feature interpretability, a key concern with deep learning medical applications. We would of course prefer that any method or tool we ever apply in a medical setting is fully interpretable, such that we, humans, can always justify and track why the model predicts what it does. In this setting, we would really want to know which diagnoses are most indicative of pancreatic cancer risk?

Even though deep learning models are generally far less interpretable than more explicit modeling approaches, there are lots of active research directions that aim to reliably map deep learning reasoning to human-interpretable steps.

Integrated gradients is one such interpretability approach to estimate feature attribution: how much each input feature contributes to predicting outcome. Here, the attribution of each feature is estimated by cumulating its gradients along all trajectory points from input to the baseline, summed up across all trajectories for pancreatic cancer patients.

For each time prediction interval, the input features with highest total attribution scores represent the diagnosis codes with highest (isolated) associations with pancreatic cancer (Figure 15). Even though these top codes are hardly informative by themselves, they are nevertheless interesting to interpret, especially by comparing their contribution to prediction across the different time prediction intervals (early cancers vs. late cancers).

As expected, most of these codes are no strangers to clinical practice. Some codes are quite specific (e.g., “non-insulin dependent diabetes mellitus”), while others are very general (e.g., “medical observation and evaluation for suspected diseases and conditions”). For trajectories that end closer to cancer diagnosis (0-6 and 6-12 months), the top features are directly related to pancreatic cancer, whereas for cancers diagnosed later, the top features can represent early sign of the disease (if only they were more specific!).

One interesting example is type 2 diabetes, a well-known risk factor for pancreatic cancer (see also Figure 7). In the Danish dataset (Figure 15 b), the importance of non-insulin dependent type 2 diabetes in predicting pancreatic cancer decreases with getting closer and closer to the pancreatic cancer diagnosis. Such temporal patterns of feature attribution among early and late cancers could indicate causal or mechanistic roles for selected diagnostic codes (causal as they are most important longer time before diagnosis). This is in stark contrast with “unspecified jaundice” for example, which is a known clinical occurrence closely preceding a pancreatic cancer diagnosis.

Subscribe now

5. Thoughts

So, what do deep learning and pancreatic cancers have in common? This study shows how deep learning provides a feasible and actionable solution to a real clinical problem that will likely improve pancreatic cancer survival: early detection of pancreatic cancers. That’s why this paper is so important.

Predicting health outcomes (in this case, a pancreatic cancer diagnosis) from health records has historically been one of the most fruitful use-cases of deep learning in the medical domain, as large volumes of health records data already exist and are straight-forward to mine. However, one of the main criticisms of this type of learning is that the insights obtained are often not clinically actionable. This criticism doesn’t hold true for the present paper.

From a modeling perspective, explicitly encoding the temporal dependencies among disease codes (rather than using them without this information) helps to improve the predictive model, and is something that most previous models do not attempt. What’s more, applying Transformers to trajectories of disease codes, and thinking of disease codes in medical history as “words in sentences” is appealing.

One of the most interesting results is comparing performance when all disease codes are included into the model (better), versus when only known pancreatic cancer risk factors are considered. This shows that (at least some of) the medical history that we would have likely labeled as un-informative is in fact informative to a certain extent. It also shows that the predictive relation between disease codes and pancreatic cancer diagnosis is something that we cannot yet understand and explicitly model.

Obvious open questions and future directions include:

• How about predicting other diagnoses except pancreatic cancer? The proposed deep learning framework is general enough to allow this, as no explicit pancreatic cancer domain knowledge goes into the architecture of the model. This question can therefore be answered right away.

• How would such a model perform in other healthcare settings, except the two presented in this paper? To answer this, a large corpus of training data from a new healthcare setting would need to be available. Nurses Health Study might be a suitable use-case to answer this.

All that being said, we have many reasons to be extremely excited and hopeful about the future (Figure 16). Disease codes represent not only a specific data type, but also a biased one. Disease codes are medical diagnoses, the large majority of which are generated rather indirectly, as the consequence of an external process: the person was not feeling well, hence they went to the doctor. In other words, diagnoses are only a narrow and specific window into the intricate workings of living humans. Most of the complexity of living remains largely untapped, with an immense potential to be discovered and utilized.

Such modeling frameworks as the one discussed in this post can be enhanced in several ways with complementary data types, which, in my opinion, hold a true potential to boost prediction accuracy to levels we cannot even imagine.

For example: how about integrating the bloodwork values from regular check-ups (most of which represent unbiased sampling) into this prediction model?

How about data on heart rate, cardio fitness, or sleep quality, which are continuously recorded by wearables nowadays?

How about glucose values from continuous glucose monitors?

Even though this perfect multi-modal dataset does not exist today, it will certainly exist at some point in the near future. This is only the beginning of this journey. There is so much about ourselves that we don’t yet know, and so much to discover.

6. Summary

• Pancreatic cancer is a disease with poor survival, usually diagnosed very late

• Early detection is well positioned to make a difference in pancreatic cancer survival, provided that high-risk people can be identified for screening and surveillance before being diagnosed with the disease

• This study proposes a deep learning method to predict the risk of pancreatic cancer occurrence based on longitudinal sequences of disease codes (temporal patient clinical records)

• The method is applied on two large datasets: the Danish Public Health Registry data (6.2 million patients) and the US-Veterans data (3 million patients)

• Transformers is the best-performing modeling architecture

• Performance is higher when using the entire patient medical history, as compared to only disease codes previously known to be associated with pancreatic cancer. This indicates that seemingly uninformative medical events do in fact hold predictive information

• Based on this methodology, a realistic and clinically-implementable surveillance program would propose for follow-up patients at least 59 times (Danish data) and 33 times (US Veterans data) more likely to be diagnosed with pancreatic cancer in the following year than a non-informative prediction based on disease incidence alone

• Lots of open questions and exciting avenues for improving prediction and future discoveries

Cancer is a disease we all know too well and a main reason why people die. Why is this? Cancer is not a new problem. In fact, it exists since thousands of years. It has been studied to unimaginable levels of detail. Clinicians are seeing the same disease over and over again. Then, why do people still die of cancer?

1. How things used to be

It’s 1938, and Public Health Services are advising people that detecting and treating cancers early will save their lives (Figure 1).

“Early Cancers Can Be Cured”

Now fast-forward nowadays. We hear the exact same core message from the Public Health Services of our times, gradually and consistently backed up by more and more data (Figure 2).

How and why is public health advice from 1938 so relevant today, 85 years later? Does this mean we made no progress in curing cancer?

Certainly not. Let me show you why.

2. What we accomplished

By zooming into cancer medical care in 1938, we quickly realize how rudimentary it actually was (Figure 3). Take radical mastectomy as an example: a truly invasive procedure developed by William Stewart Halsted at the end of the 19th century, in which the whole breast, all lymph nodes under the arm, and the chest wall muscle were removed. Radical mastectomies were still the norm following breast cancer diagnosis until 1970s. Nowadays, such surgery is almost never done, and has been replaced by local removal of the tumor (lumpectomy), followed or preceded by multiple options of additional therapies.

Now, let’s better understand the real impressive transformations that cancer science and medical practice have undergone in the last decades. I’ve chosen 10 such milestones (1938-2023) to emphasize below (Figure 4). This list is by absolutely no means exhaustive or even comprehensive, rather it is meant to contour the conceptual backbone of this progress (see here a more detailed list of cancer milestones).

1. 1941 - Hormonal Therapy for Prostate Cancer. Charles Huggins discovers that lowering testosterone production by removing the testicles or by administering estrogen leads to shrinkage of prostate tumors. Hormonal therapy continues to be a main treatment for prostate cancer nowadays.

2. 1958 - Combination Chemotherapy. James F. Holland, Emil Freireich, and Emil Frei propose that using multiple chemotherapy agents simultaneously will make it more difficult for tumors to become resistant. Their hypothesis builds upon generalizing the current state of art in tuberculosis treatment, in which the combination of antibiotics was more efficient than single agents. Nowadays, combination chemotherapy is a widespread standard of care in cancer centers around the world.

3. 1978 - Tamoxifen. An anti-estrogen drug, originally developed as a birth control treatment, is approved by the FDA for the treatment of estrogen receptor positive (ER+) advanced breast cancer. In the following years, Tamoxifen will be shown to also be effective for early disease. Tamoxifen is a targeted treatment, i.e. it specifically targets only a particular type of cancerous cells (unlike chemotherapy). It is delivered as a tablet, and is currently the gold standard treatment for ER+ breast cancers, which make up the overwhelming majority (80%) of all newly-diagnosed breast cancers.

4. 1998 - Trastuzumab. FDA approves Herceptin (Trastuzumab), a monoclonal antibody that inhibits the tumor-stimulating effects of the overexpressed amplicon HER2, for the treatment of metastatic HER2+ breast cancers (and later on also for treating early-stage HER2+ cases). Therapies involving Trastuzumab (usually in addition to chemotherapy) currently remain the gold standard in treating HER2+ breast cancers.

5. 2001 - Imatinib. FDA approves Imatinib (Gleevec) for the treatment of chronic myeloid leukemia (CML), transforming a cancer death sentence into a manageable condition. Back in 1960s, Peter Nowell and David Hungerford (then a researcher at the University of Pennsylvania) found that CML patients had an abnormally short chromosome (later named the Philadelphia chromosome). Later on, Nora Heisterkamp at NCI identifies the reason why the Philadelphia chromosome was shorter: when this chromosome forms, two genes get fused together, forming the BCR-ABL fusion protein. Further, when this fusion happens in blood cells, it causes CML. The drug Gleevec, first conceptualized by Brian Drucker at Oregon University, blocks this fusion, hence only targeting faulty cells that display it. 5 years after the first phase 1 clinical trial of Imatinib (1998), 98% of treated CML patients were disease-free. Nowadays, Imatinib is considered a potential curative treatment: if a CML patient is in remission after 2 years of Imatinib treatment, their life expectancy is the same as a person who doesn’t have cancer. In May 2001, this monumental discovery makes it on the cover of TIME magazine (Figure 5).

6. 2006 - Gardasil. On a completely different line of research, the first vaccine for the prevention of a human cancer is approved by the FDA. Gardasil, originally developed by researchers at the University of Queensland in Australia, protects against infections from four Human Papillomavirus (HPV) strains: HPV16 and 18, causing 70% of cervical cancers, and HPV 6 and 11, causing 90% of genital warts. Harald zur Hansen, who contributed to proving that genital HPV infections can causally lead to cervical cancer, is awarded the Nobel Prize in 2008 for this discovery. The approval of this vaccine marks an essential milestone also because it is a therapy for preventing cancer occurrence, rather than treating existing tumors. The current form of this vaccine is Guardasil 9, which protects against 5 additional HRV types.

7. 2011 - Ipilimumab. In all fairness, the last decade belongs to immunotherapy, which aims to boost the natural body’s defense system to eliminate cancerous cells. Even though the idea of unleashing the immune system against cancer goes back hundreds of years, most substantial immunotherapy clinical advances have been very recent, facilitated by technological progress. Ipilimumab is the first FDA-approved immune checkpoint inhibitor, which are monoclonal antibodies that release the breaks of the immune system to trigger immune-mediated anti-tumor responses. Immune checkpoints themselves are molecules that work to maintain immune tolerance, but are often used by tumor cells to their advantage, to evade immunosurveillance. Ipilimumab activates the immune system by targeting CTLA-4, a co-inhibitory molecule expressed on T cells which negatively regulates T cell activation.

8. 2014 - Pembrolizumab (Keytruda) is the first PD-1 immune checkpoint inhibitor approved by the FDA. PD-1 is expressed on the surface of several immune cell types (including T cells) and acts as a negative regulator of immune response. Its ligand PDL-1 is regularly expressed in normal tissues and regulates immune tolerance by suppressing T cell-mediated immune proliferation and cytokine secretion when binding with PD-1. The interaction between PD-1 and PDL-1 diminishes the activity of the immune system and plays an essential role in preventing autoimmune disease. But, tumor cells hijack this mechanism and abnormally express PDL-1 to escape immune surveillance. Antibodies targeting PD-1 or PDL-1 have been approved for the treatment of multiple cancer types. In 2017, a conceptual shift in cancer immunotherapy takes place, and Keytruda is approved for general use in any tumor with a certain genetic profile (based on tumor mutational burden), rather than a specific site. The blockage of the PD-1 pathway has been very successful in the clinic. Currently, most cancer types can benefit from this treatment.

9. 2017 - Kymirah (Tisagenlecleucel) is approved by the FDA as the first chimeric antigen receptor (CAR) T cell therapy. CAR T-cells are T cells which are genetically modified and expanded ex vivo (outside the patient) to mount personalized tumor attacks, by targeting tumor-specific mutations displayed on the surface of cancer cells (called neoantigens). These engineered fighters are then infused back into the patient, with the idea of selectively attacking the cells they were designed to target. A pioneering early-stage study in late 2022 coupled non-viral CAR-T therapy with CRISPR editing, showing promise for increased clinical efficacy of CAR-T products. As of today, multiple cancer types have shown sustained and real clinical responses to immunotherapy, even though 3 main drawbacks still exist 1) immunotherapy is very expensive; 2) response rates remain limited; 3) there is no consistent satisfactory understanding of clinical or genetic factors predicting these responses.

10. 2020 - AI in cancer. This decade belongs to artificial intelligence, which is quickly transforming the way we understand and interact with our own biology. Large AI models can now efficiently learn the image features separating cancerous lesions from normal ones from large amounts of labeled data. Such models are already deployed in medical centers across the world to assist with clinical cancer diagnosis. The pace of change is extremely rapid, and powerful AI models will soon be able to 1) assist in effective and efficient drug design; 2) accurately predict patient-specific personalized responses to various types of therapies; 3) flag high-risk conditions for clinical management, with the goal of preventing cancer from occurring in the first place. As more and more data is being produced and labeled, the applications and utility of such models in healthcare will skyrocket.

    ---

Ok, this all sounds truly impressive. But how much does it matter in the end? How do all these great therapeutic discoveries factor in for the one metric that we humans care about most: the time we have to live?

The harsh truth is that, despite great progress in cancer research, the drop in death rate from cancer is not as impressive. This becomes even more apparent when compared with what used to be the main source of death in the developed world (cardiovascular disease), for which the drop has been twice as large (Figure 6; I chose to display Switzerland as a best-case scenario for healthy lifestyle and optimal healthcare).

There are lots of complex reasons to explain the differences in magnitudes, some more intuitive than others. Nevertheless, it does leave us thinking. And it begs the question: why is progress against cancer so difficult?

3. Why cancer still wins

Cancer is a relentless evolutionary process. Cancer evolves to escape most therapies we use against it, no matter how potent. Every tumor is its own different word (Figure 7), and everything cancers want is to live. Fighting an evolutionary desire to live is hard. Cancer is a living process, but not a foreign one. It’s a living process living its own parallel life, as part of ours. Gradually, it hijacks and overtakes our own inner working mechanisms. This is why, conceptually, cancer is a very hard problem. Because, in a sense, the enemy is ourselves. It’s all different pieces of the same puzzle.

Now let’s get back to where we started. 1938: early cancers can be cured.

85 years later: can we now cure all early cancers? Unfortunately not, and it’s also really, really complicated. But, do we understand better today than in 1938 why this is? Definitely.

And this takes us so much closer to where we want to be.

4. Summary

• Many impressive medical discoveries have happened in cancer research in the past century.

• But, the survival rate from cancer has only modestly increased.

• Cancer is a very hard problem because of its evolutionary nature, always finding new ways to survive.