Modelling moving objects has become a topic of increasing interest in the area of video databases. Two key aspects of such modelling are spatial and temporal relationships. In this paper we introduce an innovative way to represent the trajectory of a single moving object and the relative spatio-temporal relations between multiple moving objects. The representation supports a rich set of spatial topological and directional relations. It also supports both quantitative and qualitative user queries about moving objects. Algorithms for matching trajectories and spatio-temporal relations of moving objects are designed to facilitate query processing. These algorithms can handle both exact and similarity matches. We also discuss the integration of our moving object model based on a video model in an object-oriented system. Some query examples are provided to further validate the expressiveness of our model.