Tensor decompositions are becoming increasingly important in processing images and videos. Previous methods, such as ANDECOMP/PARAFAC decomposition (CPD), Tucker decomposition (TKD), or tensor train decomposition (TTD), treat individual modes (or coordinates) equally. Their results do not contain a natural and hierarchical connection between a given tensor and its lower-order slices (e.g., a video and its frames). To overcome the practical limitation of existing tensor decomposition methods, we propose an innovative High-Order Orthogonal Decomposition (HOOD) for arbitrary order tensors. HOOD decomposes a given tensor using orthogonal linear combinations of its lower-order slices. Each orthogonal linear combination will be further decomposed. In the end, it decomposes the given tensor into orthogonal rank-one tensors. For object detection and recognition tasks in high-resolution videos, HOOD demonstrated great advantages. It is about $100$ times faster than CPD with similar accuracy detection and recognition results. It also demonstrated better accuracy than TKD with similar time overhead. HOOD can also be used to improve the explainability because the resulting eigenimages visually reveal the most important common properties of the videos and images, which is a unique feature that CPD, TKD, and TTD do not have.