m40 · AI for cardiac imaging and substrate: atrial segmentation, scar and fibrosis, automated chamber reconstruction
Education only. Not for diagnosis, treatment or any clinical decision.
In 60 seconds
Before and during an ablation, images give the mapping system a shape to work on: the left atrium outlined on CT or MRI, scar or fibrosis marked in its wall, or a shell built from intracardiac echo. Networks now draw these outlines. In an open benchmark of left atrial segmentation on late-gadolinium MRI, the best method reached a Dice score of 93.2% and a mean surface distance of 0.7 mm (Xiong et al. 2020). Whether such a number is good enough depends on the structure and on the use. This module derives the metrics that judge a segmentation (Dice, Hausdorff distance, average surface distance and normalized surface Dice), shows why the same 1-mm boundary error costs a thin wall far more Dice than a blood pool, and introduces the concordance index used for risk models built on images. The clinical correlate follows MRI-defined fibrosis from an association with outcome to a randomized trial of fibrosis-guided ablation (Marrouche et al. 2014, 2022).
From the field
- Benchmarks set the language. The 2018 Left Atrium Segmentation Challenge gave 27 teams 154 late-gadolinium MRI volumes with labels from three experts; methods that first found a region of interest and then segmented it did better than single networks (Xiong et al. 2020). In the Medical Segmentation Decathlon, the left atrium task had only 20 labelled training volumes, and methods were ranked by Dice and by a surface metric with a 4-mm tolerance for the heart (Antonelli et al. 2022). The winning method, nnU-Net, configures itself for each new data set (Isensee et al. 2020).
- Open tools from industry and academia. MONAI, an open-source PyTorch framework for medical imaging, was initially led by NVIDIA and King’s College London; it ships U-Nets, Dice losses and a loader for the Decathlon data (Cardoso et al. 2022). Its labelling tool, MONAI Label, was shown on the Decathlon left atrium, where an expert needed about 10 minutes to outline one atrium by hand (Diaz-Pinto et al. 2022). Lab part F uses MONAI’s U-Net when it is installed.
- Shells from intracardiac echo. One centre compared a deep-learning module that builds the left atrial shell from intracardiac echo with its earlier manual contouring: total procedure time 122.5 against 129.0 minutes, and 1-year freedom from AF 80.0% against 81.4% (not significantly different), in a before-and-after design (Zou et al. 2026).
- Thin walls. CT measurements in 60 patients with persistent AF found a mean left atrial wall thickness of 1.89 mm, ranging from 0.5 to 3.5 mm (Beinart et al. 2011). At that scale one voxel is a large share of the wall (demo m40-d2).
- From images to risk. A deep survival model on contrast-enhanced cardiac MRI predicted arrhythmic sudden death with a concordance index of 0.83 on internal validation and 0.74 on an independent test set (Popescu et al. 2022). Stanford and King’s College groups combined CT shape, deep image features and clinical data to predict AF recurrence after ablation (retrospective, nested cross-validation) (Razeghi et al. 2023).
- How to test an imaging AI. Stanford’s EchoNet-Dynamic segmented the left ventricle on echo videos with a Dice of 0.92 and released 10,030 annotated videos (Ouyang et al. 2020). Members of that team then ran a blinded, randomized trial of AI against sonographers for the first read of ejection fraction (He et al. 2023): a design that imaging AI in EP has rarely had.
Derivations
Symbols: \(P\) and \(T\) the predicted and true regions (sets of voxels or points), \(\lvert\cdot\rvert\) volume or area, \(\partial P\) and \(\partial T\) their surfaces, \(d(x, \partial T)\) the distance from a point to a surface, \(\delta\) a boundary offset (mm), \(S\) and \(V\) surface area and volume, \(R\) a sphere radius, \(t\) a wall thickness, \(\tau\) a distance tolerance.
Overlap
The Dice coefficient \(2\lvert P\cap T\rvert/(\lvert P\rvert+\lvert T\rvert)\) was derived in m39. Some papers report the Jaccard index (intersection over union) \(J = \lvert P\cap T\rvert/\lvert P\cup T\rvert\) instead. Since \(\lvert P\cup T\rvert = \lvert P\rvert + \lvert T\rvert - \lvert P\cap T\rvert\),
\[ \mathrm{Dice} = \frac{2J}{1 + J}, \qquad J = \frac{\mathrm{Dice}}{2 - \mathrm{Dice}} \tag{1}\]
so the two rank methods identically, but \(J\) is always the smaller number: \(J = 0.8\) is a Dice of 0.889. Convert before comparing papers.
Overlap is blind to where errors lie. A stray island far from the atrium costs only its own volume; a boundary error along a vein costs a lot of Dice for a small structure. The Decathlon organizers noted that Dice suits large structures and is unstable for very small ones, where a few voxels change the score (Antonelli et al. 2022; Reinke et al. 2024).
Key takeaways
- Dice and Jaccard carry the same information; convert with Equation 1 before comparing.
- Overlap says how much, not where.
Distances between surfaces
Distance metrics compare the two surfaces. The Hausdorff distance is the largest distance from any point of one surface to the other:
\[ \mathrm{HD}(P, T) = \max\left\{ \max_{p \in \partial P} d(p, \partial T),\; \max_{q \in \partial T} d(q, \partial P) \right\} \tag{2}\]
Because it is a maximum, one island sets it. The 95th-percentile Hausdorff distance (HD95) replaces each maximum by the 95th percentile and ignores the worst 5% of points; toolkits differ in whether they pool the two directions first, so check which one a paper used. The average symmetric surface distance (ASSD) averages every distance:
\[ \mathrm{ASSD} = \frac{\int_{\partial P} d(p, \partial T)\, dp + \int_{\partial T} d(q, \partial P)\, dq}{\lvert\partial P\rvert + \lvert\partial T\rvert} \tag{3}\]
On voxels the integrals become sums over surface voxels. The normalized surface Dice (NSD) counts the share of both surfaces that lie within a tolerance \(\tau\) of the other:
\[ \mathrm{NSD}_\tau = \frac{\lvert\{p \in \partial P : d(p, \partial T) \le \tau\}\rvert + \lvert\{q \in \partial T : d(q, \partial P) \le \tau\}\rvert}{\lvert\partial P\rvert + \lvert\partial T\rvert} \tag{4}\]
It asks the clinical question directly: how much of the outline is close enough? The tolerance is a choice; the Decathlon used 4 mm for the heart, set with clinicians (Antonelli et al. 2022).
Exact
Assumptions
- Two-dimensional discs and circles. Dice from the disc areas; HD, ASSD and NSD from Equation 2, Equation 3 and Equation 4 with distances between the outlines, weighted by arc length and integrated by the midpoint rule on 3,600 points (720 for the curves). The values match masks rasterized at 0.05 mm (
tests/test_m40_imaging.py). - The island is a disc whose nearest edge lies the chosen gap beyond the outer of the two main outlines, so it never overlaps them.
- Default radius 20 mm and island size are illustrative. The default tolerance, 4 mm, is the one the Decathlon used for its heart task (Antonelli et al. 2022).
- What would change the result: real errors are uneven along the outline, and 3-D surfaces weight regions by area.
Key takeaways
- Hausdorff distance reports the worst point; one island sets it.
- HD95 and ASSD summarize typical error; NSD reports the share within a chosen tolerance.
- State the tolerance and the percentile convention with every surface metric.
What a boundary error costs
Suppose a segmentation is right in shape but its whole boundary sits \(\delta\) too far out (or in). The volume between the two surfaces is about \(S\lvert\delta\rvert\), and Dice is \(1 - \lvert P \,\triangle\, T\rvert/(\lvert P\rvert+\lvert T\rvert)\), where \(\triangle\) is the symmetric difference. To first order in \(\delta\),
\[ \mathrm{Dice} \approx 1 - \frac{\lvert\delta\rvert\, S}{2V} \tag{5}\]
The loss scales with the surface-to-volume ratio \(S/V\): large, round structures lose little; thin or branched ones lose a lot. Two exact cases show the range. For concentric spheres of radii \(R\) and \(R + \delta\),
\[ \mathrm{Dice} = \frac{2\min(R, R+\delta)^3}{R^3 + (R+\delta)^3} \approx 1 - \frac{3\lvert\delta\rvert}{2R} \tag{6}\]
and for a wall of thickness \(t\) shifted by \(\delta\) along its normal,
\[ \mathrm{Dice} = \max\!\left(0,\; 1 - \frac{\lvert\delta\rvert}{t}\right) \tag{7}\]
A 1-mm error on a sphere of radius 25 mm (illustrative, about the size of a blood pool) leaves Dice at 0.94. The same error on a wall 1.89 mm thick, the mean measured by Beinart et al. (2011), leaves 0.47. A Dice reported for the left atrial blood pool and one reported for its wall or for scar inside the wall are therefore not on the same scale.
Exact
Assumptions
- Exact formulas: Equation 6 for the sphere, the same formula in two dimensions for a long cylinder grown radially, and Equation 7 for the wall. The readout adds the first-order estimate of Equation 5 for the sphere.
- The default wall thickness, 1.89 mm, and the slider range, 0.5 to 3.5 mm, are the mean and range measured on CT by Beinart et al. (2011). The sphere radius (25 mm) and cylinder radius (5 mm) are illustrative.
- What would change the result: real errors are not uniform, and real atria are neither spheres nor slabs (lab part E measures real atrial labels).
Key takeaways
- For a uniform boundary error, Dice falls with surface-to-volume ratio (Equation 5).
- The same millimetre costs a thin wall far more than a blood pool.
- Compare Dice only between structures of similar shape and size.
Risk models from images
Some imaging models predict when an event will happen rather than outline a structure. They are judged by Harrell’s concordance index (Harrell et al. 1982). Take every pair of patients \((i, j)\) where \(i\) had the event at time \(t_i\) earlier than \(j\)’s follow-up time \(t_j\); the model should give \(i\) the higher risk \(r_i\):
\[ C = \frac{\sum_{i,j} \mathbb 1[t_i < t_j]\, e_i \left(\mathbb 1[r_i > r_j] + \tfrac12\, \mathbb 1[r_i = r_j]\right)}{\sum_{i,j} \mathbb 1[t_i < t_j]\, e_i} \tag{8}\]
where \(e_i = 1\) if patient \(i\) had the event. \(C = 0.5\) is chance and \(C = 1\) is perfect ordering. \(C\) measures ranking only: a model can order patients well and still predict the wrong absolute risk, which calibration measures such as the integrated Brier score address. The deep survival model of Popescu et al. (2022) reported both: \(C\) of 0.83 and 0.74 and integrated Brier scores of 0.12 and 0.14 on internal validation and the independent test set.
Key takeaways
- \(C\) is the share of comparable pairs a risk score orders correctly.
- Report calibration as well as \(C\).
- Expect \(C\) to drop on an independent cohort.
Clinical correlate
This section describes published findings only. It is not guidance for patient care, and it has not yet been reviewed by an EP physician.
Fibrosis on MRI: from association to a randomized trial. In the DECAAF cohort, atrial fibrosis was quantified on delayed-enhancement MRI at a core laboratory before AF ablation in 15 centres (Marrouche et al. 2014). Of 329 enrolled patients, 57 were excluded for poor image quality. Each 1% more fibrosis was associated with more recurrence (unadjusted hazard ratio 1.06 per 1%), and adding fibrosis to clinical covariates raised the C statistic from 0.65 to 0.69.
DECAAF II then randomized 843 patients with persistent AF in 44 centres to pulmonary vein isolation (PVI) plus ablation of MRI-defined fibrosis, or PVI alone (Marrouche et al. 2022). Atrial arrhythmia recurred in 43.0% and 46.1% (hazard ratio 0.95, 95% CI 0.77 to 1.17), not significantly different. The safety composite occurred in 2.2% of the fibrosis-guided group and in none of the PVI group, including 6 ischemic strokes in the fibrosis-guided group. A later analysis found that ablation lesions did not uniformly produce scar on 3-month MRI, with large differences between centres (Akoum et al. 2024).
Shells and workflow. For an AI-built left atrial shell from intracardiac echo, the published comparison is with an earlier period of manual contouring at one centre (Zou et al. 2026). It reports shorter procedure and left atrial dwell times and similar 1-year freedom from AF.
In the terms of this module and m45:
- An image-derived quantity can predict outcome without being a good target. DECAAF showed an association; DECAAF II tested a treatment built on it and found no difference in recurrence.
- Quantification depends on segmentation. Fibrosis lives in a wall about 2 mm thick (Beinart et al. 2011), where a 1-mm boundary error removes half the overlap (Equation 7).
- Before-and-after designs confound the tool with time. Operators, workflows and patient mix change between periods; randomization, as in the echo trial of He et al. (2023), separates them.
Visual summary
Schematic
Assumptions
- Schematic: no quantitative claim. Metrics follow Equation 1 to Equation 8.
How experts think
Before trusting an imaging model in an EP workflow, ask:
- Which structure, at what size? A blood pool, a vein, a wall or scar inside the wall: the same error gives very different Dice (Equation 5, demo m40-d2).
- Which metrics, with which settings? Dice for overlap, HD95 or ASSD for typical distance, NSD at a stated tolerance; Jaccard converted with Equation 1.
- Where are the errors? Pulmonary veins, the appendage and the mitral plane are where outlines disagree (lab parts C and D); the benchmark of Tobon-Gomez et al. (2015) standardized these regions before scoring.
- What does the error do downstream? For a shell used with a map, measure the distance from touched points to the shell (lab part D), not only Dice.
- Was a decision tested, or only an association? DECAAF and DECAAF II answer different questions (Marrouche et al. 2014, 2022).
One sentence to keep: a segmentation score means something only for a named structure, a named metric and a named use.
Cheat sheet
| Quantity | Result |
|---|---|
| Dice and Jaccard (Equation 1) | \(\mathrm{Dice} = 2J/(1+J)\), \(J = \mathrm{Dice}/(2-\mathrm{Dice})\) |
| Hausdorff distance (Equation 2) | Largest point-to-surface distance, both directions; HD95 uses the 95th percentile |
| ASSD (Equation 3) | Mean surface-to-surface distance, both directions |
| NSD (Equation 4) | Share of both surfaces within \(\tau\) of the other |
| Boundary offset (Equation 5) | \(\mathrm{Dice} \approx 1 - \lvert\delta\rvert S/(2V)\) |
| Spheres (Equation 6) | \(2\min(R,R+\delta)^3/(R^3+(R+\delta)^3) \approx 1 - 3\lvert\delta\rvert/(2R)\) |
| Shifted wall (Equation 7) | \(\max(0, 1 - \lvert\delta\rvert/t)\) |
| Concordance (Equation 8) | Share of comparable pairs ordered correctly; 0.5 is chance |
Studies in this module
| Study | What was done | Evidence |
|---|---|---|
| Xiong et al. (2020) | Left atrium on late-gadolinium MRI, 27 teams | Open benchmark |
| Antonelli et al. (2022) | Ten segmentation tasks, including the left atrium | Open benchmark |
| Popescu et al. (2022) | Arrhythmic death risk from scar on MRI | Internal validation and independent test set |
| Zou et al. (2026) | AI-built left atrial shell from intracardiac echo | Single-centre before-and-after |
| Marrouche et al. (2022) | Strategy: MRI-fibrosis ablation plus PVI vs PVI | Randomized trial |
| He et al. (2023) | AI vs sonographer first read of ejection fraction (echo) | Blinded randomized trial |
Rules of thumb
- Always pair an overlap metric with a distance metric.
- Report surface metrics per region (body, veins, appendage) when the use depends on them.
- Find the region of interest first: in the left atrial benchmark, two-stage networks did best (Xiong et al. 2020); lab part E shows why.
Interview questions
Try each question before you open its model answer. In a real interview, say which assumptions you are making.
Q1. A paper reports an intersection over union of 0.80; another reports a Dice of 0.86. Which is better?
Model answer
Convert first (Equation 1): \(J = 0.80\) is a Dice of \(1.6/1.8 = 0.889\), higher than 0.86. Then check that both used the same structure, data set and label protocol; otherwise the comparison is still not fair.
Q2. A segmentation has Dice 0.95 but a Hausdorff distance of 30 mm. What happened?
Model answer
Most likely a small false island or a stray protrusion far from the structure. It costs little volume, so Dice barely moves, but the Hausdorff distance is a maximum and reports the worst point (Equation 2; demo m40-d1). Look at HD95, the island itself, and whether a largest-component step would remove it.
Q3. When would you report HD95, ASSD or NSD?
Model answer
HD95 for a near-worst-case distance that ignores a few outliers; ASSD for the typical error over the whole surface (Equation 3); NSD when there is a clinically meaningful tolerance, for example the distance within which a shell is usable for guidance (Equation 4). State the percentile convention and the tolerance, since toolkits differ.
Q4. What does NSD at 4 mm of 0.95 mean?
Model answer
95% of the two surfaces, counted together, lie within 4 mm of the other surface. It says nothing about how far the remaining 5% are; pair it with HD95 or the Hausdorff distance. The Decathlon used 4 mm for its heart task (Antonelli et al. 2022).
Q5. Why does the same 1-mm error give Dice 0.94 for a blood pool and 0.47 for the atrial wall?
Model answer
Dice loss scales with surface-to-volume ratio (Equation 5). A sphere of radius 25 mm loses about \(3 \times 1/(2 \times 25) = 6\%\) (Equation 6); a wall 1.89 mm thick shifted by 1 mm keeps only \(1 - 1/1.89 = 0.47\) (Equation 7). Thin structures need distance metrics or a much finer grid.
Q6. Can you compare the Dice of a left atrial blood-pool model with that of a scar model?
Model answer
No. Scar lies inside a wall about 2 mm thick (Beinart et al. 2011) and forms small, irregular regions, so its surface-to-volume ratio is much higher, and Equation 5 predicts a lower Dice for the same boundary error. Compare each against its own baseline, such as agreement between two human raters on the same images.
Q7. How would you validate an automatically built atrial shell for use with a mapping system?
Model answer
Compare with a reference shell on held-out patients using Dice, HD95, ASSD and NSD at a tolerance agreed with users; report errors per region (veins, appendage, mitral plane), where outlines differ most (Tobon-Gomez et al. 2015). Then measure the distance from catheter contact points to the shell (lab part D). Finally, test the workflow: time, safety and outcome, ideally randomized.
Q8. A survival model has a concordance index of 0.74. What does that tell you, and what not?
Model answer
In 74% of comparable pairs, the patient with the earlier event had the higher predicted risk (Equation 8). It says nothing about calibration (whether a predicted 10% risk means 10%) or about benefit from acting on the score. In Popescu et al. (2022), \(C\) fell from 0.83 on internal validation to 0.74 on the independent test set, and the authors note that the two cohorts differ.
Q9. DECAAF found fibrosis predicted recurrence; DECAAF II found fibrosis-guided ablation did not reduce it. Is that a contradiction?
Model answer
No. A marker can predict outcome without being a target whose ablation changes outcome (Marrouche et al. 2014, 2022). The trial also found more safety events in the fibrosis-guided group, and a later analysis showed that lesions did not uniformly become scar (Akoum et al. 2024). Association and intervention are different questions.
Q10. What could bias a before-and-after comparison of AI-built and manually built shells?
Model answer
Anything else that changed between periods: operator experience (learning curve), other workflow or equipment changes, patient selection, and follow-up practice. The study design of Zou et al. (2026) compares a later period with an earlier one, so differences cannot be attributed to the AI alone. Randomization or a stepped design, with blinded outcome assessment, would separate them.
Q11. What made the AI-versus-sonographer echo trial a strong design?
Model answer
Studies were randomized to an AI or a sonographer first read, cardiologists who finalized the reports could not tell which they had (blinding index 0.088), and the primary endpoint was the share of studies whose ejection fraction the cardiologist changed by more than 5% (He et al. 2023). It also reports what it did not test: long-term patient outcomes.
Q12. In lab part E, the same voxel classifier does well in a tight box and badly in a loose one. Why, and what does that say about real pipelines?
Model answer
Local intensity features cannot tell the left atrium from other blood pools; in a tight box there are few other blood pools, in a loose box there are several, and the largest component may be the wrong one. Real pipelines need a localization step, and its errors count. The left atrial benchmark found that two-stage networks (localize, then segment) did best (Xiong et al. 2020). The tight box in the lab is taken from the label, so it is an optimistic “oracle” first stage.
Code lab
The lab computes segmentation metrics on shapes with known answers, segments synthetic atria with a threshold and a voxel classifier, measures what the errors mean for points on the wall, and repeats the key measurements on 20 real left atrial labels. Parts A to D run in under a minute.
| Part | What you do | Data |
|---|---|---|
| A | Metrics with known answers: Equation 6, Equation 7, an island and the Hausdorff distance | synthetic |
| B | Synthetic atria: body, veins, appendage, a bright neighbour, bias and noise | synthetic |
| C | Threshold against a voxel classifier; Dice, HD95, ASSD and NSD; body against veins | synthetic |
| D | From mask to map: distance from points on the true wall to the predicted shell | synthetic |
| E | Real left atrial labels: shape, boundary errors, islands, and a classifier in a tight or loose box | MSD Task02_Heart |
| F | Optional: a 2-D U-Net in PyTorch (MONAI’s implementation if installed) | synthetic |
- Python notebook: labs/python/m40_lab.ipynb. Run it from the course repository after
pip install -r requirements.txt, or open it in Google Colab. - Data for part E: the 20 labelled training volumes of the Medical Segmentation Decathlon heart task (Antonelli et al. 2022), from the 2013 Left Atrial Segmentation Challenge (Tobon-Gomez et al. 2015), licence CC BY-SA 4.0. The archive is downloaded from the MONAI consortium’s AWS Open Data bucket and checked against the MD5 value MONAI publishes; nothing derived from it is stored in this repository. Without internet access, part E is skipped.
- Part F runs only if PyTorch is installed; it uses MONAI (Cardoso et al. 2022) when available.
Each part ends with checks: lines that pass with the code as given. If you change something and a check fails, that is information, not an error to hide.
Self-check
Short checks you can do in your head. Open each answer after you have tried it.
S1. A Jaccard index of 0.5. What is the Dice?
Answer
\(2 \times 0.5/1.5 = 0.67\) (Equation 1).
S2. A sphere of radius 20 mm is segmented 1 mm too large all round. Estimate Dice.
Answer
About \(1 - 3/40 = 0.925\) from the first-order form; the exact value is 0.927 (Equation 6).
S3. A wall 2 mm thick is segmented 1 mm off along its normal. Dice?
Answer
0.5 (Equation 7).
S4. The prediction equals the truth plus one voxel 30 mm away. What happens to Dice and to the Hausdorff distance?
Answer
Dice barely changes; the Hausdorff distance becomes 30 mm (Equation 2).
S5. Every point of both surfaces lies within 2 mm of the other surface. What is NSD at 2 mm?
Answer
1 (Equation 4).
S6. A risk score is pure noise. What concordance index do you expect? And if it ranks patients in exactly the wrong order?
Answer
About 0.5; and 0 (Equation 8).
Literature figures
These figures are worth looking at alongside this module. We link to them instead of copying them: some licenses do not allow reuse here, and others we could not confirm. Each entry says what to look for and where this module explains it.
- Ten tasks, one benchmark. Antonelli and colleagues, Fig. 1, the overview of the ten Decathlon tasks and the difficulty each was chosen for, including the small heart data set [Antonelli et al. (2022); free full text]. This module: From the field; lab part E.
- A survival curve per patient. Popescu and colleagues, Extended Data Fig. 2, predicted survival curves for one patient who had arrhythmic sudden death and one who did not, drawn against Kaplan–Meier and Cox-model curves [Popescu et al. (2022); free full text]. Compare with what the concordance index does and does not measure (Equation 8, interview Q8).
- A trial flow for an imaging AI. He and colleagues, the study flow figure, which shows the echocardiograms excluded for image quality before randomization [He et al. (2023); free full text]. Compare with the before-and-after design of the clinical correlate (interview Q10, Q11).
Further reading
- Metrics. Maier-Hein et al. (2024) (Metrics Reloaded) and Reinke et al. (2024) (metric pitfalls).
- Benchmarks and methods. Tobon-Gomez et al. (2015), Xiong et al. (2020), Antonelli et al. (2022) and its preprint with per-task results (Antonelli et al. 2021), Ronneberger et al. (2015) (U-Net), Isensee et al. (2020) (nnU-Net).
- Open tools. Cardoso et al. (2022) (MONAI), Diaz-Pinto et al. (2022) (MONAI Label).
- Imaging and risk. Popescu et al. (2022), Razeghi et al. (2023).
- Fibrosis and ablation. Marrouche et al. (2014), Marrouche et al. (2022), Akoum et al. (2024).
- Echo AI from Stanford. Ouyang et al. (2020), He et al. (2023).
- AI competency areas. Machine learning (segmentation). See the AI competency map.