Abstract : HD video content represents a tremendous quantity of information that cannot be easily handled by all types of devices. Hence the scalability issues in its processing have become a focus of interest in HD video coding technologies. In this chapter, we focus on the natural scalability of hierarchical transforms to tackle video indexing and retrieval. In the first part of the chapter, we give an overview of the transforms used and then present the methods which aim at exploring the transform coefficients to extract meaningful features from video and embed metadata in the scalable code-stream. Statistical global object-based descriptor incorporating low frequency and high-frequency features is proposed. In the second part of the chapter, we introduce a video retrieval technique based on a multiscale description of the video content. Both spatial and temporal scalable descriptors are proposed on the basis of multi-scale patches. A statistical dissimilarity between videos is derived using Kullback-Leibler divergences to compare patch descriptors.