This dissertation provides an overview and analysis of existing methods of tensor decomposition and describes a non-redundant tensor decomposition in terms of which we define the rank of a tensor. A tensor is a multidimensional or p-way array of scalars. Decompositions of tensors have applications in psychometrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere when we analyze order-p data with p ≥ 2. The simplest case of tensor decomposition is singular value decompositions (SVD) when the order p equals 2, in this case SVD transforms data space into parametric space preserving cardinality. Two widely used tensor decompositions can be considered to be extensions of the SVD without preserving cardinality: CAN-DECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-1 tensors, and the Tucker decomposition is a higher-order form of principal component analysis (PCA). We present a tensor decomposition that includes SVD as a particular case, describes a tensor as a set of variables, defines an upper bound for the rank of tensors, and does not have redundancy as in the cases of CP and Tucker decompositions.