Network component analysis (NCA) and other methods based on the NCA model have become powerful bioinformatics tools to reconstruct underlying regulatory networks and recover hidden biological processes. However, due to the existence of experimental noises in microarray data and false information in network connectivity data (e.g., ChIP-on-chip binding data, motif information, etc.), it still remains challenging to reconstruct gene regulatory networks for real biomedical applications such as human cancer studies. In this paper, we model the relationship between the genes that share the same transcription factors (TF) from the angle of regression. We propose a statistic called outlier sum testing the conditional significance of the target genes. A Gibbs strategy is utilized in order to estimate the marginal value of outlier sum from its conditional function. Based on the outlier sum statistic we are able to extract the true target genes that carry information about transcription factor activities (TFAs) from the whole population. As a proof-of-concept, we demonstrated the efficiency and robustness of the proposed method on both simulation data and yeast cell cycle data.