The advances in the application of Virtual Environments (VE) in medicine help to improve the conventional methods of delivering health services by enhancing and complementing traditional approaches of medical education, diagnostic techniques, intraoperative assistance, pre-post operative services etc. Virtual Reality (VR) and Augmented Reality (AR) are two forms of VEs that revolutionize medical training and fill the deficiencies in the traditional teaching/learning methods in medicine. Although presenting the unreality in a form of blended virtuality is exceedingly accepted in medicine, it brings many challenging but intriguing and intertwined multi disciplinary problems that are required to be addressed.

This thesis proposal sorts out the problems of facilitating the VE systems particularly in the AR and the VR settings that deal with algorithmic problems originated from integration of different components particularly related to surgical simulations. More specifically, the work presented here includes extensive literature reviews on both the VR and the AR based simulators in medicine. Moreover, what we would like to bring into the readers' attention is the two specific problems namely registration in AR and soft-tissue deformation in VR, as well as their solutions and analysis.

First of all, in the subject of registration, we aim to resolve the alignment problem of the real world object on to its virtual counterpart for an AR simulator. In conventional registration procedure which is manually performed, the entire process is usually tedious and can be erroneous. Therefore, the need for a greatly simplified registration process that eliminates human involvement and turns the developed registration scheme into an indispensable tool for Vicon based AR simulators in medicine was apparent, specially due to the fact that the registration of the objects with Vicon optical motion tracking system is not noted sufficiently in the literature, and that makes our study even more unique regarding the application field of surgical instrumentation.

The second achievement of the project is simulation of response of a soft-tissue to an applied force otherwise known as soft-tissue deformation. We have two schemes to demonstrate soft-tissue behavior. One is a mass-spring model and the other, graphical processing unit (GPU) based geometric model. Our contribution to mass-spring model is the perspective on developing and analysis of data structures. The analysis hereby gives us an intuitive and quantitative assessment of the performance of our data structures. This analysis on developed data structures can help mass-spring model users to select the most efficient data structure for their soft-tissue simulations.

Our other contribution in the area of simulation of soft-tissue behavior to an applied force is the GPU based geometric method. The developed technique provides very large polygonal objects' deformations in real-time by revealing the GPU's excessive parallel computation power. The algorithm allows simulating deformable bodies' behaviors in real-time interactivity for high resolution models even with the low-cost personal desktop computer equipped with a GPU. The method completely relocates the computational burden from Central Processing Unit (CPU) to the GPU and frees the CPU resources which results in compact and modular structures. The approach and final results implies that the GPU based approach can be very well suited for the VR simulators.