In this paper we present automatic vehicle detection and tracking system for aerial surveillance. In the existing frameworks they apply either region based or sliding window based of vehicle detection in aerial surveillance. These method have highly depends on the color segmentation, a lot of miss detections on rotated vehicles and high computational complexity. So we propose a new method for vehicle detection using dynamic Bayesian networks. We consider features including vehicle colors and local features. For vehicle color extraction, we utilize a color transform to separate vehicle colors and non vehicle colors effectively. For edge detection, we apply moment preserving to adjust the thresholds of the canny edge detector automatically, which increases the adaptability and the accuracy for detection in various aerial images. Afterward, a dynamic Bayesian network (DBN) is constructed for the classification purpose. We convert regional local features into quantitative observations that can be referenced when applying pixelwise classification via DBN. For future work, performing vehicle tracking on the detected vehicles can further stabilize the detection results. Automatic vehicle detection and tracking could serve as the foundation for event analysis in intelligent aerial surveillance systems.