Realistic visualization of internal organs Organs react realistically in real time to: - User interactions
- Environmental restrictions
Organs react to typical surgeon’s gestures through geometric and topological modifications
First generation: First generation: - Only deal with geometric nature of human anatomy
Second generation: - + permit physical interactions with anatomy
- Include needle-type, exploration-type, catheter installation-type simulators as well as simulators that permit training in only one task and full simulators
Third generation: - + consider functional nature of organs
Physical Modeling Reduction of Computing Time Collision Detection Example Systems
Data set consists of about 180 slices of frozen human tissue that has been put through CT scan Data set consists of about 180 slices of frozen human tissue that has been put through CT scan - Enhance contrast
- Apply edge detection
- Semi-automatic deformable models → binary images
- Stack images to form 3D binary image [Montagnat, 1997]
Better than marching cubes – avoids “staircase effects Better than marching cubes – avoids “staircase effects Developed by Delingette to represent 3D objects [Delingette, 1994] Adaptable (figure to right) Working on a method to extract liver models from CT images
How physically realistic the model is correlated with how realistic force feedback is How physically realistic the model is correlated with how realistic force feedback is Model deforms with surgeon’s motion Contact force may be computed from deformation Force generated back to surgeon through mechanical actuators
Method uses linear elasticity as an approximation for tissue deformation Method uses linear elasticity as an approximation for tissue deformation Let the configuration of an elastic body be defined as Ω A field of volumetric and surface forces f acts on the body so it has a new configuration Ω* We want the displacement field u which associates the initial configuration of any particle with its final configuration Use FEM – Lagrange elements of type P1 [Bathe, 1996] Formulate the problem as a linear system Where [K] is the 3n by 3n stiffness matrix and n is the number of mesh vertices (more on this in a minute)
Only thing we know is endoscope position - must use displacement not force constraints
Given some displacements between the surgical tool and the body, we can find - Force on end effecter
- Global deformation
Now we use variational formulation and Lagrange multipliers to minimize Include constraints u = u* Solving for λi gives the opposite of the necessary forces to impose the displacement u* See Appendix A [Cotin, 1999] for full derivation
In theory, this behavior is only physically correct for small displacements In theory, this behavior is only physically correct for small displacements Force feedback limits the range of deformations - Feedback force on surgeon’s hand will increase as deformation increases
Mix of linear representation and empirical results using a cylindrical piece of brain tissue Mix of linear representation and empirical results using a cylindrical piece of brain tissue [Chinsei, 1997] found that deformation depends on loading speed and is nonlinear
Physical Modeling Physical Modeling Reduction of Computing Time Collision Detection Example Systems Results and Conclusion
Number of mesh vertices has high impact Number of mesh vertices has high impact Must use speedups - Cannot make necessary calculations in real-time
Specify a set of nodes to remain fixed Specify a set of nodes to remain fixed - Don’t have to set all three dof
For every “free” node k and degree of freedom on the surface, emplace an “elementary” displacement constraint (δ) Compute the displacement of every free node n in the mesh with respect to every node k - Store as set of 3X3 tensors
Compute elementary force at each constrained node k
Must be solved 3m times where m is the total number of free nodes inside the tetrahedral mesh Can take anywhere from a few minutes to several hours
For any n where k ≠ n, the relation between n and k is For any n where k ≠ n, the relation between n and k is Superposition may be used to find the total displacement of a node but some modifications must be made
Use tensors of deformation found in preprocessing to generate a vector of modified constraints Use tensors of deformation found in preprocessing to generate a vector of modified constraints where and
From this, we can find the displacement of any node From this, we can find the displacement of any node The force that must be applied to each node k to produce these displacements is
Computing times for a realistic looking liver model: Computing times for a realistic looking liver model:
Physical Modeling Physical Modeling Reduction of Computing Time Collision Detection Example Systems Results and Conclusion
Work discussed so far uses bounding boxes with a hash table We know about these so lets move on to a new problem – simulating the folds of the intestines
Goal is simulator to allow doctors to practice a surgery that involves pulling and folding the intestines [Raghupathi L. et. al., 2003] Goal is simulator to allow doctors to practice a surgery that involves pulling and folding the intestines [Raghupathi L. et. al., 2003] Real problem here is self-collsions Complicated by tissue called mesentery - Connects small intestine and blood vessels
Resting position: Resting position: - Intestines look like folded curves lying in a cylinder
- Mesentery is defined as line segments connecting folded intestine to the axis of the cylinder
Mechanical model uses masses and springs
Model intestines like cylinders Model intestines like cylinders “Active pairs” - Local minima satisfying certain distance threshold
- Updated every time step
N additional random pairs of segments also generated every time step - These are tested and thrown out if they are over the threshold or already represent a minimal pair
Complexity would be too high for real-time without approximation Complexity would be too high for real-time without approximation Don’t consider mesentery-mesentery interactions Adaptive convergence - Replace segment S1 by closest neighbor S to S2 and then replace S2 with neighbor closest to S
Physical Modeling Physical Modeling Reduction of Computing Time Collision Detection Example Systems Results and Conclusion
The Generic Real Time Surgery Simulator [Monserrat et al., 2003] The Generic Real Time Surgery Simulator [Monserrat et al., 2003]
Allows user to select tools and organs needed Allows user to select tools and organs needed Systems contains modeling parameters for a variety of organs - Mass-spring model
- Boundary element based model (BEM)
Tools: Tools: - Loading organs
- Establishing input points for instruments
- Associating different physical properties with organs
- Establishing boundary conditions
- Linking tissues
- Adding special tissues
- Associating textures to organs
Takes a scene and allows user to train Takes a scene and allows user to train User can have interaction with organs: User can exchange instruments User is assessed at the end based on how many incorrect actions were taken
Use 450 MHz Pentium III with 256 MB memory Use 450 MHz Pentium III with 256 MB memory Computational Costs:
For good visual image 15Hz refresh rate For good haptic stimulus 500 Hz refresh rate Use a PC cluster to solve this Cost of force feedback devices makes simulator 4X more expensive than without
Surgery aims to extract cataract and replace it with intraocular lens [Agus et al., 2006] Surgery aims to extract cataract and replace it with intraocular lens [Agus et al., 2006] Training is important Simulation allows: - Flexibility
- Gradual increase in difficulty
- Exposure to rare events
- Quantification of performance
Phacoemulsification: breaking hardened lens into fragments and removing them with a small sucker using the phacoemulsificator Phacoemulsification: breaking hardened lens into fragments and removing them with a small sucker using the phacoemulsificator Create z-shaped corneal tunnel Capsulorhexis: removing the anterior capsule to uncover the upper surface of the crystalline
Decoupled simulation: Decoupled simulation: - Fast subsystem for surgical instrument tracking and slower one for visual feedback
- Slow subsystem does global simulation and interaction of devices and eye
- Slow subsystem can be further broken into individual visual effects
Force feedback is useless in this surgery - Must use eye globe visualization
- Conjugate gradient to minimize energy constraints gives equilibrium position
- Rotate to reduce deformation
Use triangular mesh with a mass-spring network mapped over it Use triangular mesh with a mass-spring network mapped over it Mass particles may be anchored, scripted or free Weak springs simulate sticking effects Solve ODE using semi FSAL (First Same as Last) Velocity found using implicit method and feedback on position is computed explicitly Correction routine applied after each step to correct position and velocity as required by constraints Tearing – breaking overextended springs
Lens as collection of simplices Lens as collection of simplices - Tetrahedron mesh with particles placed at barycenters
- Links connecting particles maintained for rendering and determining independent particles
Photoemulsificator modeled by eroding particles in a zone of influence Employ Russian roulette scheme to decide which particles to erode Idea is to replace energies by geometric constraints and forces by distance from current position to goal Each connected subset of points is associated with a point cloud - Shape matching with undeformed rest state to determine goal positions
Physical Modeling Physical Modeling Reduction of Computing Time Collision Detection Example Systems Results and Conclusion
Tried penalty and constraint methods but stability of the system was reduced Tried penalty and constraint methods but stability of the system was reduced Instead alter displacement velocities to avoid penetration
Interpolating: Interpolating: Need force f’ = f so we have: New velocities are: Substituting we get:
Solving for f gives: Solving for f gives: Condition for avoiding penetration takes radii into account: The force required to change the positions of the endpoints to satisfy these conditions is:
Marco Agus, Enrico Gobbetti, Giovanni Pintore, Gianluigi Zanetti, and Antonio Zorcolo. Real-time Cataract Surgery Simulation for Training. In Eurographics Italian Chapter Conference. Eurographics Association, 2006. Marco Agus, Enrico Gobbetti, Giovanni Pintore, Gianluigi Zanetti, and Antonio Zorcolo. Real-time Cataract Surgery Simulation for Training. In Eurographics Italian Chapter Conference. Eurographics Association, 2006. K.-J. Bathe, Finite Element Procedures. Prentice Hall, 1996. K. Chinsei and K. Miller, “Compression of Swine Brain Tissue Experiment In Vitro,” J. Mechanical Eng. Laboratory, pp. 106-115, 1997. S. Cotin, H. Delingette, and N. Ayache. “A Hybrid Elastic Model allowing Real-Time Cutting, Deformations and Force-Feedback for Surgery Training and Simulation.” The Visual Computer, 16(8):437-452, 2000. Cotin, S.; Delingette, H.; Ayache, N., "Real-time elastic deformations of soft tissues for surgery simulation," Visualization and Computer Graphics, IEEE Transactions on , vol.5, no.1, pp.62-73, Jan-Mar 1999 H. Delingette, ”Simplex Meshes: A General Representation for 3D Shape Reconstruction,” Technical Report 2214, INRIA, Mar. 1994. Y.C. Fung, Biomechanics-Mechanical Properties of Living Tissues, second ed. Springer-Verlag, 1993. Carlos Monserrat, Oscar López, Ullrich Meier, Mariano Alcañiz Raya, M. Carmen Juan Lizandra, Vicente Grau: GeRTiSS: A Generic Multi-model Surgery Simulator. IS4TH 2003: 59-66 J. Montagnat and H. Delingette, “Volumetric Medical Images Segmentation Using Shape Constrained Deformable Models,” Proc. First Joint Con5 CVRMed-MRCAS ’97, J. Troccaz, E. Grimson, and R. Mosges, eds. Mar. 1997. M. Moore and J. Wilhelms, “Collision Detection and Response for Computer Animation,” Computer Graphics (SIGGRAPH ’88), vol. 22, pp. 289-298, Aug. 1988. Laks Raghupathi, Laurent Grisoni, Fran?ois Faure, Damien Marchal, Marie-Paule Cani, Christophe Chaillou, "An Intestinal Surgery Simulator: Real-Time Collision Processing and Visualization," IEEE Transactions on Visualization and Computer Graphics, vol. 10, no. 6, pp. 708-718, November/December, 2004.
Do'stlaringiz bilan baham: |