Intra-procedural navigation · Mixed reality · Biopsy

CarnaLife Holo MedNav.
The needle guided along a planned path

The patient's CT or MR study is superimposed onto their body in a mixed reality headset, while the biopsy needle and the ultrasound probe are tracked without any attached markers. The physician plans the trajectory before the puncture and sees, in real time, how far the needle tip deviates from it.

Book a demonstration See how it works

87%.: target hit rate with navigation, against 33% with ultrasound alone
-53%.: shorter preparation and procedure time
3.41 mm.: upper bound of the ultrasound–CT fusion error at 95% confidence
0.: markers attached to the needle or the ultrasound probe

The figures come from studies published in Cancers (2024) and Scientific Reports (2026), carried out on phantoms and tissue models. The full texts are available in the “Publications” section.

The problem

A flat image, a three-dimensional body and a needle you cannot see

Percutaneous biopsy is one of the most frequently performed diagnostic procedures. The physician inserts a needle through the skin, guided by an ultrasound, CT or MR image. Each of these methods has its own limitations - and they all share one: the image is flat, the patient's body is not.

The operator has to combine, in their head, a two-dimensional image with the real position of the needle deep in the tissue. If the needle leaves the image plane, it simply disappears. The result is repeated puncture attempts, and every additional puncture increases the risk of complications, blood loss and patient discomfort.

Ultrasound
Advantages
Fast, radiation-free, shows the needle live and lets the operator avoid vessels with Doppler.
Limitations
The image is flat. The needle is often poorly visible, and the result depends on the operator's experience and on finding a good acoustic window.
Computed tomography
Advantages
Excellent contrast and spatial resolution, usable in almost any region of the body.
Limitations
Radiation dose, a higher complication risk and no real-time preview - the image only appears after another scan.
Magnetic resonance
Advantages
The best soft-tissue contrast, no ionising radiation, vessels visible without contrast agents.
Limitations
Difficult access to the lesion, awkward verification of the collected sample and the need for dedicated needles.

MedNav does not replace any of these methods. It adds the layer that has been missing: information about where exactly the needle sits in space relative to the lesion — including when it is not visible in the image.

How it works

Four steps from the scan to a guided puncture

The whole workflow relies on what is already in the room: the patient's scan, the ultrasound machine and a HoloLens 2 headset. No mechanical arms, no external tracking cameras.

01.
The scan becomes a 3D model
A standard CT or MR study in DICOM format is reconstructed by CarnaLife Holo into a volumetric model. The physician can rotate it, scale it, slice it and measure on it — not on a monitor, but in the space around them.
02.
The model is superimposed on the patient
The patient is scanned with radiological markers attached to the skin. The operator indicates the same landmarks in the imaging data and on the patient's body, and the system computes the transformation that lays the hologram over the patient at true scale.
03.
The needle trajectory is planned before the puncture
The physician marks a target point inside the lesion and an entry point on the skin. A straight line appears, which can still be checked in the 3D view and on 2D slices — to make sure the needle path avoids critical structures.
04.
The system guides the hand in real time
Rings appear around the planned trajectory, and guiding lines run from both ends of the needle to that path. The operator's task is very simple: shorten those lines to zero. Colours change as the alignment improves.

Planning a biopsy trajectory in the CarnaLife Holo software — Step 03. Trajectory planning in CarnaLife Holo: on the left the volumetric reconstruction of the CT study with the needle path marked, on the right the same plan on a sagittal slice. The first cross is the entry point, the second the target.

Indicating radiological markers in the imaging data — Step 02. Patient registration: the operator indicates the same radiological marker in the imaging data and on the patient's body. On this basis the hologram is fitted to real-world dimensions.

Navigation interface: trajectory rings and the tracked needle — Step 04. The view from the headset: green rings mark the planned trajectory and the blue line is the detected needle position. The task is to pass the needle through the centre of the rings.

Biopsy needle navigation seen through the mixed reality headset — Guidance in the ultrasound variant. The green line is the planned trajectory, the blue one the current needle position. The white leading lines show the deviation; the operator's task is to shorten them to zero.

Markerless tracking

The instrument recognized by its shape, not by a sticker

Most mixed reality navigation systems require QR codes or retro-reflective spheres to be attached to the instrument. That works in a laboratory, but in a procedure room it means extra equipment, calibration before every procedure and the constant need to keep the marker in the camera's field of view.

MedNav recognises the needle and the ultrasound probe purely from the images produced by the cameras built into the headset. A neural network locates the instrument in the picture and makes an initial estimate of its pose; depth camera data then corrects that estimate by fitting the instrument model to the real point cloud. There is no separate calibration step.

Ultrasound probe pose estimation before and after depth-based refinement — The effect of the depth-based refinement. On top the output of the neural network alone, below the same measurement after refinement. Green marks the reference pose and blue the algorithm's estimate; after refinement the two almost coincide.

Every refinement step reduces the error

Below, the mean orientation error of the ultrasound probe in degrees, measured separately for an unobstructed view and for the case where the operator's hand occludes the instrument. After all the steps the error falls below one third of the starting value — and it practically stops depending on whether the hand blocks the view.

Baseline10.79° / 12.94°
+ RANSAC4.87° / 9.81°
+ instrument mask4.20° / 4.86°
+ Kalman filter3.51° / 3.85°
+ EMA2.94° / 3.21°

unobstructed viewinstrument occluded by the hand

73 ms: full ultrasound probe pose estimation
25 ms: needle position estimation
197–230 ms: latency from instrument motion to the image in the headset
0.20 mm: accuracy of the reference tracker the system was compared against

Image fusion

The live ultrasound image inside a three-dimensional CT or MR study

Since the system knows exactly where the ultrasound probe is, it also knows which plane of the patient's body the probe is imaging. Cutting the volumetric CT or MR model with that same plane places both images precisely on top of each other.

This is an important difference from classic fusion, which aligns images based on their content - and therefore fights ultrasound noise and can be unreliable. Here it is enough to know where the probe is, so the method works regardless of the anatomical region and shows the result immediately.

The radiological phantom used in the study — The radiological phantom with imitation lesions used to measure fusion accuracy. The white crosses are the radiological markers used for superimposition.

Fusion of the ultrasound image with the lesion contour from CT — The ultrasound image with the lesion contour from the CT study overlaid in red. The operator sees the lesion even when it is barely outlined in ultrasound alone.

The two-dimensional ultrasound image positioned inside the three-dimensional CT model — The same fusion seen from a different angle: the two-dimensional ultrasound image stands exactly at the place in the three-dimensional CT model that it is imaging.

1.97 ± 0.89 mmmean target registration error between ultrasound and CT

3.41 mmvalue below which the error stays with a probability of 0.95

400measurements taken across twenty acquisitions from different locations

Study results

What changed in the measurements

The results below come from two independent series of tests: on phantoms and on an ex vivo tissue model, with physicians of varying experience taking part. In each case the same procedure was compared with and without navigation.

Target hit rate
with MedNav87.1%
without navigation32.9%
Ex vivo tissue model, spheres 8–20 mm in diameter, punctures 7–11 cm deep. Mixed reality navigation against ultrasound alone.
Procedure completed on the first attempt
with MedNav70%
without navigation20%
Ten physicians, deformable phantom. Fewer punctures mean a lower complication risk, less blood loss and less patient discomfort.
Average number of punctures
with MedNav1.4
without navigation2.5
Here lower is better. Every extra puncture is another risk factor for complications.

53%.

shorter procedure time

Average across ten physicians: 53 seconds with navigation against 1 minute 57 seconds without it, counting preparation and the puncture itself on the phantom.

100% / 96%.

material collection efficiency

A 5 cm lesion was hit in all 24 attempts and a 2 cm lesion in 23 out of 24, with access from different sides and at depths of 3.6–10 cm.

16 cm.

verified target depth

The needle position after the procedure was confirmed with a repeat CT scan. The error grows with depth, but stays within an acceptable range even for the deepest targets.

Clinical value

Where navigation makes the biggest difference

Spatial awareness instead of imagination
With classic ultrasound the operator has to combine a flat image with the three-dimensional position of the needle in their head. Here both are visible in the same place, at true scale.
Punctures outside the image plane
The hardest punctures are those where the needle leaves the ultrasound plane and disappears. Navigation shows its position even when it is not visible in the image.
Lesions next to critical structures
When the needle path runs close to vessels or the pleura, a planned and monitored trajectory reduces the operator's cognitive load at the most demanding moment.
Lesions barely visible in ultrasound
Fusion with an earlier CT or MR study makes it possible to target a lesion that is practically invisible in the ultrasound image alone.
A training tool
In the tissue-model test, a person with no procedural experience achieved, with navigation, a success rate more than two and a half times higher than with ultrasound alone.
No extra tower in the room
Tracking runs on the cameras built into the headset. There is no external tracker to position and no calibration before each procedure.

Publications

Every figure on this page comes from peer-reviewed publications

The system was developed by MedApp S.A. together with AGH University of Krakow and with clinicians. The full texts of both articles are available below.

What still lies ahead

An honest account of the limitations, as described by the authors of the studies. Each of them is the subject of further work.

Superimposing the data on the patient still requires manually indicating the markers. Automating this step is the subject of further work.
The fusion study was performed on a rigid phantom. In clinical practice the images may drift apart because of breathing, cardiac motion, instrument pressure or patient repositioning.
Depth sensing relies on the built-in time-of-flight camera, which is sensitive to lighting, reflections and object material. Headsets with stereo cameras should improve this component.
The studies covered one needle type and one ultrasound probe. Smaller instruments and instruments in sterile sheaths require a separate assessment.

Would you like to see MedNav in your own suite?

Please get in touch - we will show trajectory planning and needle guidance on a phantom, using your own DICOM data, and tell you which procedures benefit from navigation the most.

CarnaLife Holo MedNav.
The needle guided along a planned path

A flat image, a three-dimensional body and a needle you cannot see

Ultrasound

Computed tomography

Magnetic resonance

Four steps from the scan to a guided puncture

The scan becomes a 3D model

The model is superimposed on the patient

The needle trajectory is planned before the puncture

The system guides the hand in real time