This paper presents the results of a series of simulation studies examining the performance of algorithms for integrating Inertial Navigation Systems (INS) with Vision-based Navigation (VISNAV) systems. Two Extended Kalman Filtering (EKF) approaches called loose and tight integration are compared. The tight INSVISNAV integration approach fuses INS information with camera generated pixel measurements and uses non-linear time and measurement update equations. The loose integration approach fuses INS information with the position and attitude solution generated using the camera pixel measurements. The loose integration uses a non-linear time update equation but linear measurement equation. The results show that although tight integration yields a more accurate navigation solution, it has a tendency to diverge under certain conditions. It is shown that the conditions which lead to divergence are related to: (1) Unfavorable relative geometry between the camera and feature points used to construct the VISANV solution and (2) Large errors in the position and attitude solution about which the tight integration measurement equations are linearized. This latter condition can occur when VISNAV updates are infrequent or spaced far apart in time. On the other hand, loose integration shows better stability and robustness in the unfavorable geometry conditions which lead to tight integration divergence. Furthermore, since the loose measurement update equations are linear, it is insensitive to infrequent or widely spaced measurement updates.