Real-time vehicle tracking in ITS (Intelligent Transportation Systems) is highly desirable. In this paper, we propose a hardware-based tracking system to achieve real-time vehicle tracking. The proposed tracking system employs two innovative designs. One is that a tracking algorithm based on motion estimation is adopted. Motion estimation is achieved by full-search block matching algorithm. The other innovation is that motion estimation is implemented in hardware, which greatly reduces computational time compared to the traditional software approach. Once the motion vectors are obtained, the remaining tracking process is implemented in software. Experiments have validated the tracking algorithm and simulations have shown that the proposed tracking system can achieve realtime tracking up to 40 frames per second.