Augmented Surgery
A stereoscopic viewer that allows the surgeon to have information about the patient before his eyes and to guide his hand with high precision
The current context
Healthcare is a sector that is undergoing profound transformations thanks to continuous technological evolutions. Surgeries, once considered complex, are now performed thanks to the use of new tools and new technologies.
Telemedicine and Connectivity allow you to receive various types of data from patients remotely.
An increasingly personalized approach to therapeutic strategies based on the individual characteristics of single patients.
According to a study published in Technology Review, for example, thanks to Big Data, it is possible to collect enough data to allow doctors to know 12 months in advance and with a 98% confidence rate if a particular drug will cause side effects to a particular patient.
AI and Big Data in the Healthcare sector
The use of Machine Learning generates greater value from the data, as in addition to carrying out descriptive analyses (past reporting), it allows to identify correlations and express “predictions” with inductive reasoning, typical of the human mind.
These analysis tools could identify new risk factors both in the onset of pathologies, in the onset of complications, and in directing therapeutic choices.
The further sophistication in examining the data is prescriptive analysis: machine learning software capable of explaining the rules underlying the predictive models (therefore, NOT black boxes), allow what-if simulations to understand whether and how, through the modification of some factors, the outcomes can be improved, thus selecting the optimal behaviours.
Augmented Surgery
Mixed Reality
Mixed Reality in support of surgical activity is an innovative industrial research and experimental development project.
It consists of a stereoscopic viewer that allows the surgeon to have in front of his eyes, without having to look away from the operating field, the result of the processing of the medical images acquired before and during the surgery.
The virtual information, perfectly aligned with the patient’s anatomy, offers the surgeon an “X-ray view” to guide his hand with high precision. The patient’s vital parameters (heartbeat, blood oxygenation, etc.) are also projected on the viewer, always remaining in the surgeon’s field of view. These parameters are projected together with the parameters related to the patient’s clinical history enclosed and stored within an Electronic Medical Record system capable of interacting with the view system.
The image offered to the observer is in fact an “unicum” recreated from a synthesis carried out by real time algorithms starting from the data acquired simultaneously from stereoscopic cameras (real scenario) and from clinical images (virtual scenario) obtained in the pre- and intra-operative phase.
Medical Record
The purpose of the activity is the use of the electronic medical recording a non-static but dynamic way, as a producer and repository of data from the monitoring tools that can be dynamically interfaced with the Augmented Reality system and that is also accompanied by Machine Learning algorithms for Early Warnings and a Business Intelligence system for the management of synthetic indicators to be reproduced on the vision system. In addition to the consultation of the information, it will, in fact, be possible to actively act on the folder thanks to the possibility of inserting and storing in the operating diary all the activities that have been performed including those recognized by the classifiers, appropriately implementing the Mixed Reality paradigm.
Furthermore, the use of Visual Data Discovery techniques to improve the surgeon’s visualization of the health data contained in the Patient Medical Record will represent a substantial improvement for Business Intelligence in the healthcare sector.
The advantages deriving from the use of these techniques in a mixed scenario of Augmented Reality are linked to the possibility of exploring and better organizing the data for a better organization of information by providing greater immediacy and reactivity of analysis in accordance with the operational scenario.
Machine Learning
The project also envisages the creation of a ” real-time” predictive to support the activity carried out by medical professionals during surgery in the surgery in order to show “warning signals” based on the patient’s vital parameters.
The creation of this model involves the use of Artificial Intelligence algorithms, in particular Machine Learning.
This is made possible thanks to the historical data, inherent to the patients, collected during the surgeries. This “input” information consists of the vital parameters collected with a certain temporal frequency during the surgery.
The Process
The use of Artificial Intelligence and Big Data will carry out in real time a personalized parameterization of the most significant vital parameters in relation to the patient’s pathology and the intervention based on millions of cases.
DOCTOR IN THE OPERATING ROOM
In real time, the surgeon visualizes on the lenses those vital parameters that need more attention
BIG DATA
Various technologies analyse databases and medical records with affinity to the patient’s medical history
MACHINE LEARNING ALGORITHMS
Advanced Machine Learning algorithms calculate data close to the limits to be displayed on smart glasses
ETL PROCESSES
Extraction, Transformation and Loading Systems, conform the data for input into the AI system
The Benefits
Risk stratification occurs today mainly through the extraction of historical data that generate weak clinical validity. The new challenge is represented by the use of Big Data and business analytics systems to activate risk stratification models based on real health status, on the integrated use of multiple sources and on collaboration between professionals (care team and data manager, in a data mining process).
Possibility of scanning large databases that cannot be analysed with traditional statistics
Possibility of carrying out descriptive, predictive and prescriptive analyses
Possibility to identify new variables and new risk factors
Possibility of implementing a phenotype/genotype integration, for a personalized medicine
Minimize drugs complications by increasing safety
The Partners
Lead Partner
EOS s.r.l.
ICT Company specialized in industry 4.0 and delivery technologies
Partner
CINI
The reference point of the national academic research of Informatics and Information Technology
Partner
CRdC Tecnologie s.c.a.r.l.
Regional centre of competence that interfaces between research and production
Partner – Startup
Kratos srl
ICT Company specialized in Healthcare
Lender
Campania Region
Support to Campania companies in carrying out feasibility studies (Phase 1) and technology transfer projects (Phase 2) consistent with RIS3
CUP B63D18000460007
