Objective:Risk assessment associated with the implementation of the semi-automatic time-lapse system Geri® Assess 2.0 for morphokinetic analysis of embryos, from fertilisation to blastocyst, using artificial intelligence to predict implantation quality and capacity; with integration of software reliability results using the Euro-GTP II framework.
Methods:The risk assessment è was conducted according to the three phases foreseen by EuroGTP II, taking into consideration the results of the study “Investigation of the reliability of semi-automatic annotation by the Geri time-lapse system” (Reprod Biomed Online, 2022), including identification of the technological novelty, risk analysis and determination of the extent of studies required.
Results:The EuroGTP II analysis identified critical risks related to non-detection of anomalies (R=16) and over-dependence on software (R=20). The experimental study showed automatic detection rates between 0% and 94.4%, with sensitivity 68.2–94.4% and specificity 63.8–97.3%. Automatic annotation required manual intervention in 55.2% of the oocytes and showed a concordance with manual scoring of 42.1%, which increased to 66.0% after correction. The software showed important limitations in detecting “direct cleavage” (in some cases 0%) and“reverse cleavage”.
Conclusions: Geri Assess 2.0 is a useful tool as an aid to morphokinetic assessment, but cannot be used as a primary decision-making system, so verification with the EURO GPT II system results in a moderate risk. The EuroGTP II approach enables systematic risk management by defining the necessary mitigation measures to ensure quality and safety in clinical use. Mandatory review of all automatic annotations by qualified embryologists è manual verification è essential for risk mitigation.
Introduction
Substances of human origin (SoHO) are being used in an increasing number of innovative clinical applications, including time-lapse technologies applied to medically assisted procreation (MPA). The rapid development of new blood, tissue and cell (BTC) components, as well as new preparation protocols, makes it necessary to adopt standardised processes to identify, quantify and manage the potential risks associated with their clinical implementation.
In this context, the EuroGTP II tool, developed by the EDQM, is a risk assessment system designed to:
- determine whether a BTC or process represents a novelty compared to the standard (Phase 1);
- evaluate associated risks through a combination of probability, severity and detectability (Phase 2);
- define the extent of studies and follow-up required to ensure safety and efficacy (Phase 3).
The introduction of the semi-automatic time-lapse system Geri Assess 2.0 for embryo morphokinetic annotation is fully among the processes requiring a EuroGTP II assessment, given the innovative nature of the technology and its potential impact on the quality of clinical outcomes. GERI Assess 2.0 è a semi-automated time-lapse system that analyses embryonic morphokinetics, from fertilisation to blastocyst, using advanced imaging and artificial intelligence. It provides a score to predict embryo quality and implantation capacity, optimising selection and improving results in IVF treatments.
Materials and Methods
Evaluation Process via lthe EuroGTP II methodology applied to the Geri Assess 2.0 system followed the three steps provided:
Phase 1 – New Technology
Geri Assess 2.0 è been classified as “new process”, requiring:
- deep local validation,
- specific staff training,
- quality system enhancement,
- defining performance criteria (performance specifications).
.
Step 2 – Risk analysis
Risks related to:
were identified and assessed
- failure to detect critical anomalies,
- excessive reliance on automation,
- variability between operators,
- incorrect timing of kinetic events,
- possible integration errors and data loss.
.
Each risk has been classified on the basis of the EuroGTP II criteria:
Probability (P), Severity (G), Detectability (R)
with risk calculation: R = P × G
.
Phase 3 – Extension Study
The phase included:
- defining the extent of validation,
- implementation of specific KPIs,
- introduction of detailed Operating Procedures,
- indication of mandatory manual review.
513 oocytes from from 34 women were analysed. The annotations made by the Geri Assess 2.0 system for ten kinetic events (the time of pronuclei (PN) appearance and disappearance, the cleavage time for two-, three-, 4-, up to 8-cell (t2–t8) cell divisions up to the annotation of the time of blastocyst formation) were compared with manual annotations made blindly by embryologists.
The following were evaluated:
- detection rates,
- sensitivityà and specificityà,
- embryonic score matching,
- ability to identify “direct cleavage” and“reverse cleavage”
- percentage of images/events that required manual review.
.
Results
EuroGTP II evaluation
Phase 1 – Evaluation of the new process (GERI 2.0)
The technology è was new to the Centre for Medically Assisted Procreation, requiring full validation.
Stage 2 – Risk assessment
- The main risks identified in Table 1 include: false negatives in clivings anomalies (R=16, critical); over-reliance on software without expert review (R=20, critical); tincorrect filling of key events such as t2, t3, tPN (R=12, high); fpositive bias, data loss, insufficient traceability, inter-operator variability (R=8–9, moderate).
|
Risk identification |
Prob. (1–5) |
Grav. (1–5) |
Rilev. (1–5) |
Risk (PxG) |
Level |
||
|
False negatives (direct/reverse cleavage) |
4 |
4 |
2 |
16 |
Critic |
||
|
False positives: incorrectly reported events |
3 |
3 |
2 |
9 |
Moderate |
||
|
Incorrect timing (t2, t3, PN) |
3 |
4 |
2 |
12 |
High |
||
|
Software dependency/non-review |
4 |
5 |
2 |
20 |
Critic |
||
|
Integration/data loss errors |
2 |
4 |
3 |
8 |
Moderate |
||
|
Variability between operators |
3 |
3 |
3 |
9 |
Moderate |
||
|
Insufficient traceability |
2 |
4 |
3 |
8 |
Moderate |
||
Table 1
Results of the experimental study
- Detection rates: 0–94.4%
- Sensitivityà: 68.2–94.4%
- Specificità: 63.8–97.3%
- Concordance with manual annotation: 42.1%
→ 66% after manual annotation - Corrections required: 55.2% of oocytes
- Direct cleavage: detection also 0%
.
· Reverse cleavage: low and variable detection
These results confirm the need for expert review to ensure accuracy and security.
Risk mitigation measures
- Mandatory review of all automatic annotations by qualified embryologists.
- Double check for embryos intended for transfer/biopsy.
- Detailed Operating Procedures for Correcting Annotations.
- Initial and periodic validation of the system on local dataset.
- Complete audit trail with logging of all changes.
- Data backup and version control.
- Regular staff training on time-lapse reading.
- monthly KPIs for monitoring concordance and error rates.
- Introduction of quality controls aimed at monitoring Critical Process Parameters (CPPs)
.
Discussion
The integration of the EuroGTP II results with the experimental study clearly shows that Geri Assess 2.0 can contribute to the standardisation of morpho-kinetic annotation, but cannot replace the embryologist’s analysis. The main critical issues are related to the detection of cleavage abnormalities (e.g. t1-t3 “direct cleavage” or t2-t1 “reverse cleavage"), which have a direct clinical impact on embryonic developmental potential.
The risk assessment was repeated using the EuroGTP II tool, systematically reviewing the critical points highlighted by the article from the expert embryologist's point of view: the process was reclassified as new and all the risk mitigation strategies listed above were applied. Based on these measures, the risk level calculated by the system è resulted low, with a residual risk of 6, as shown in Figure 1, indicating that the GERI 2.0 technology can be used safely only as a support and not as a primary decision-making tool.
Figure 1Risk assessment
The new tool also emphasises the need to accompany the introduction of the technology with a structured clinical follow-up programme, allowing for a rigorous comparison of the results obtained before and after its implementation and to verify its effectiveness over time. This evaluation must be complemented with a systematic comparison against data available in the literature to ensure that the performance obtained is consistent with morphokinetic data reported by other centres. With this new GERI 2.0 è it is possible to have under control the main embryo morphokinetic points and to standardise the laboratory KPIs ensuring an objective evaluation in the critical steps from fertilisation to blastocyst.
Conclusions
The EuroGTP II report explicitly concludes:
“The semi-automatic annotation of the Geri® Connect & Assess system can only be used safely with mandatory manual review and after documented local validation. Residual risk è acceptable as an aid, not as a primary decision-making tool.”
Bibliography
- EDQM. Guide to the Quality and Safety of Tissues and Cells for Human Application. Council of Europe; latest edition.
- EuroGTP II Consortium. EuroGTP II – Good Practices for Tissues and Cells: Risk Assessment Tool. EDQM, Council of Europe; accessed materials and guidelines.
- Garcia-Belda A et al Considerations for future modification of The Association for the Study of Reproductive Biology embryo grading system incorporating time-lapse observations. Reprod Biomed Online. 2023 doi: 10.1016/j.rbmo.2023.103570.
- Paternot G, et al. Automated and semi-automated embryo annotation systems: accuracy and clinical performance. Hum Reprod. 2013;28(3):627-33. doi: 10.1093/humrep/des427.
- Kieslinger DC et al. The inconvenient reality of AI-assisted embryo selection in IVF. Nat Med. 2024 Nov;30(11):3059-3060. doi: 10.1038/s41591-024-03289-9. PMID: 39354198.
- Armstrong S, et al. Time-lapse systems and embryo selection: systematic review and meta-analysis. Cochrane Database Syst Rev. 2019. https://doi.org/10.1002/14651858.CD011320.pub4
- Rubio I, et al. Morphokinetic markers associated with embryo viability: a multicentre evaluation. Hum Reprod. 2020. doi: 10.1016/j.fertnstert.2014.07.738
- Meseguer M, et al.The use of morphokinetics as a predictor of embryo implantation. Hum Reprod. 2011 Oct;26(10):2658-71. doi: 10.1093/humrep/der256.
- Jessica Vandame, et al. Investigation of the reliability of semi-automatic annotation by the Geri time-lapse system,Reproductive BioMedicine Online,Volume 45, Issue 1, https://doi.org/10.1016/j.rbmo.2022.02.012.
- Novo S et al. The effect of reverse cleavage on embryo development and clinical outcomes. Reproductive BioMedicine Online 2024 doi:10.1016/j.rbmo.2024.104473
