# Tensor train decomposition

## Top PDF Tensor train decomposition: ### Uniqueness of tensor train decomposition with linear dependencies

1 Introduction Canonical Polyadic Decomposition (CPD)  is one of the most used tensor decompositions in signal processing. The CPD and its variants are attractive tools due to their ability to decompose tensors into physically interpretable quantities, called factors. Its uniqueness has been studied in several state- of-art articles such as [7, 10, 5]. Uniqueness and compactness are two of the advantages that make the CPD widespread. In- deed, the CPD is usually unique under mild conditions and its storage cost grows linearly with respect to the order. Recently, tensor networks (TNs)  have been subject of increasing in- terest, especially for high-order tensors, allowing more flexi- ble tensor modelling. TNs split high-order (Q > 3) tensors into a set of lower-order tensors. Tensor train decomposition (TTD)  is one the most compact and simple TNs. Indeed, TTD breaks a high Q-order tensor into a set of Q lower-order tensors, called TT-cores. These TT-cores have orders at most equal to 3. In this sense, TNs are able to break the “curse of dimensionality”.
En savoir plus ### A continuous analogue of the tensor-train decomposition

1 (x 1 )f 2 (α 1 ,α 2 ) (x 2 ) · · · f d (α d−1 ,α d ) (x d ), (1) with r 0 = r d = 1. Analytical examples in  demonstrate how certain functions can be represented in this continuous low-rank format. In this paper we provide a computational methodology for approximating multivariate functions and computing with them in this format. We do not discretize functions on tensor-product grids and we do not a priori specify a tensor-product basis for approximation, and as a result our ap- proach overcomes the limitations described above. Furthermore, our approach is adaptive and akin to incorporating adaptive grid refinement within the context of low-rank approximations. While some related approaches [18, 19, 20] represent low-rank functions without relying on tensor-product discretizations or coefficient tensors, the firs two [18, 19] employ the canonical polyadic format and all of them rely on fixed (non-adaptive) parameterizations to learn from scattered data. To the best of our knowledge, there are no existing methods for low-rank function approximaton that adapt parameterizations (on a grid or otherwise) in addition to ranks. Overall, our contributions include:
En savoir plus ### Automated synthesis of low-rank stochastic dynamical systems using the tensor-train decomposition

eral dynamical systems, as the abstraction of the system model becomes finer and the high-level specifications become more complex, automated synthesis hits a computa- tional barrier. This barrier is known as the state explosion problem . Hence, exist- ing algorithms are either restricted to simple systems, e.g. linear dynamical systems, or they are intractable, e.g. the running time scales exponentially with increasing dimensionality of the state space. In order to handle more complex dynamics, com- putational methods typically require discretization of the state space into a regular grid, rendering the number of discrete state exponential in the dimensionality of the state space. This is known as the curse of dimensionality and is an inherent prob- lem in almost all task and motion planning problems that involve complex system dynamics. One of the few works that addresses this problem in automated synthesis is , which proposes an efficient framework for synthesizing controllers subject to high-level specifications using recent progress in tensor decompositions. The idea is to solve a series of constrained reachability problems and then composing the solutions. Unfortunately, the framework only applies to a subclass of continuous-time stochastic dynamical system which have linear partial differential equations.
En savoir plus ### TENSOR-TRAIN MODELING FOR MIMO-OFDM TENSOR CODING-AND-FORWARDING RELAY SYSTEMS

In this work, we assume a more general and practical scenario where the MIMO relaying system is operating in frequency-selective fading environment. As will be shown later, when generalizing the scheme of  to the wideband communication scenario, the tensor modeling of the received signals involves the estimation of larger quantities compared to the narrowband case. These quantities are represented by third-order channel tensors and one symbol matrix, the third dimension being associated to the frequency domain. By resorting to multicarrier modulation using orthogonal frequency division multiplexing (OFDM), we propose a new receiver design for a TCF MIMO-OFDM relaying system that is capable to solve the joint channel and symbol estimation problem. The proposed receiver fits the resulting 6-order received signal tensor to a tensor train decomposition (TTD), where the knowledge of the tensor coding structure is used to ensure identifiability of the channel tensors and the symbol matrix. It is important to note that our methodology is able to manage the case of any number of relays. To the best of our knowledge, this is the first work where the TTD approach is used to design a semi-blind receiver for a MIMO communication system.
En savoir plus ### Tensor Train Representation of MIMO Channels using the JIRAFE Method

c Department of Teleinformatics Engineering, Federal University of Fortaleza, Brazil. d Laboratoire I3S, Universit´ e Cˆ ote d’Azur, CNRS, Sophia Antiplois, France. Abstract MIMO technology has been subject of increasing interest in both academia and industry for future wireless standards. However, its performance benefits strongly depend on the accuracy of the channel at the base station. In a recent work, a fourth-order channel tensor model was proposed for MIMO systems. In this paper, we extend this model by exploiting additional spatial diversity at the receiver, which induces a fifth order tensor model for the channel. For such high orders, there is a crucial need to break the initial high-dimensional optimization problem into a collection of smaller coupled optimization sub-problems. This paper exploits new results on the equivalence between the canonical polyadic de- composition (CPD) and the tensor train (TT) decomposition for the multi-path scenario. Specifically, we propose a Joint dImensionality Reduction And Factor rEtrieval (JIRAFE) method to find the transmit and receive spatial signatures as well as the complex path gains (which also capture the polarization effects). Monte Carlo simulations show that our proposed TT-based representation of the channel is more robust to noise and computationally more attractive than available competing tensor-based methods, for physical parameters estimation. Keywords: MIMO systems, CPD, tensor train decomposition, dimensionality reduction, factor retrieval.
En savoir plus ### Tensor-train approximation of parametric constitutive equations in elasto-viscoplasticity

Keywords: parameter-dependent model, surrogate modeling, tensor-train decomposition, Gappy POD, heterogeneous data, elasto-viscoplasticity 1. Introduction Predictive numerical simulations in solid mechanics require complex material laws that involve systems of highly nonlinear Di fferential Algebraic Equations (DAEs). These models are essential in challenging industrial applications, for instance to study the e ffects of the extreme thermo-mechanical loadings that turbine blades may sustain in heli- copter engines [1, 2]. These DAE systems are referred to as constitutive laws in the material science community. They express, for a specific material, the relationship between the mechanical quantities such as the strain, the stress and miscellaneous internal variables, and stand as the closure relations of the physical equations of mechanics. Complex constitutive equations are often tuned through a set of parameters called material coe fficients.
En savoir plus ### Multidimensional Harmonic Retrieval Based on Vandermonde Tensor Train

Abstract Multidimensional Harmonic Retrieval (MHR) is at the heart of important signal- based applications. The exploitation of the large number of available measure- ment diversities for data fusion increases inexorably the tensor order/dimensionality. The need to mitigate the “curse of dimensionality” in this case is crucial. To efficiently cope with this massive data processing problem, a new scheme called JIRAFE (Joint dImensionality Reduction And Factors rEtrieval) is proposed to estimate the parameters of interest in the MHR problem, namely, the M P angular-frequencies, of the associated P -order rank-M Canonical Polyadic De- composition (CPD). Our methodology consists of two main steps. The first one breaks the high-order measurement tensor into a collection of graph-connected 3-order tensors, each following a 3-order CPD of rank-M , also called Tensor Train (TT)-cores. This result is based on a model property equivalence between the CPD and the Tensor Train decomposition (TTD) with coupled TT-cores. The second step makes use of a Vandermonde based rectified Alternating Least Squares (RecALS) algorithm to estimate the factors of interest, by enforcing the desired matrix structure. We show that our methodology has several advantages in terms of flexibility, robustness to noise, computational cost and automatic pairing of the parameters of interest with respect to the tensor order.
En savoir plus ### High-order CPD estimator with dimensionality reduction using a tensor train model

In this paper, we focus on high-order CPD models . Such models have a great interest in signal processing for blind equalization , , blind source separation , radar , wireless communications , , among many other fields of application. Most methods of factor estimation are ALS-based techniques . Unfortunately these techniques may require several iterations to converge , , and convergence is increasingly difficult when the order of the tensor increases  and it is not even guaranteed . Hence, algorithms that are stable and scalable  with the tensor order and dimensions are needed. In this work, we establish an equivalence between a Q-order CPD of canonical rank R and a tensor train (TT)  model. Note that the idea of rewriting a CPD into the TT format was briefly mentioned in . In this paper, we exploit this idea to propose a new CPD factor retrieval algorithm. Indeed, we will see
En savoir plus ### Parallel Tensor Train through Hierarchical Decomposition

the original tensor, where r is usually a very small number and depends on the application. Sequential algorithms to compute TT decomposition and TT approximation of a tensor have been proposed in the literature. Here we propose a parallel algorithm to compute TT decomposition of a tensor. We prove that the ranks of TT-representation produced by our algorithm are bounded by the ranks of unfolding matrices of the tensor. Additionally, we propose a parallel algorithm to compute approximation of a tensor in TT-representation. Our algorithm relies on a hierarchi- cal partitioning of the dimensions of the tensor in a balanced binary tree shape and transmission of leading singular values of associated unfolding matrix from the parent to its children. We consider several approaches on the basis of how leading singular values are transmitted in the tree. We present an in-depth experimental analysis of our approaches for different low rank tensors and also assess them for tensors obtained from quan- tum chemistry simulations. Our results show that the approach which transmits leading singular values to both of its children performs bet- ter in practice. Compression ratios and accuracies of the approximations obtained by our approaches are comparable with the sequential algo- rithm and, in some cases, even better than that. We also show that our algorithms transmit only O(log 2 P log d) number of messages along the
En savoir plus ### Fast BEM Solution for 2-D Scattering Problems Using Quantized Tensor-Train Format

2 Solution with tensor techniques For many years, various types of problems have been formulated using tensors instead of the classical matrix algebra . More recently, tools have been developed to treat high order tensors (dimension higher than 3) based on the implementation of low-rank techniques to e ffectively reproduce the algebraic structure of the system. In the literature, among the most prominent tensor formats for an e fficient and stable ”hollow” representation of a very large system are Tensor-Train (TT) and Hierarchical Tucker decomposition. In this work, we have chosen to apply the TT format to formulate the integral equation problem for a fast solution.
En savoir plus ### Tensor CP Decomposition with Structured Factor Matrices: Algorithms and Performance

Tensor CP Decomposition with Structured Factor Matrices: Algorithms and Performance José Henrique de M. Goulart, Maxime Boizard, Rémy Boyer, Gérard Favier and Pierre Comon Abstract—The canonical polyadic decomposition (CPD) of high-order tensors, also known as Candecomp/Parafac, is very useful for representing and analyzing multidimensional data. This paper considers a CPD model having structured matrix factors, as e.g. Toeplitz, Hankel or circulant matrices, and studies its asso- ciated estimation problem. This model arises in signal processing applications such as Wiener-Hammerstein system identification and cumulant-based wireless communication channel estimation. After introducing a general formulation of the considered struc- tured CPD (SCPD), we derive closed-form expressions for the Cramér-Rao bound (CRB) of its parameters under the presence of additive white Gaussian noise. Formulas for special cases of interest, as when the CPD contains identical factors, are also provided. Aiming at a more relevant statistical evaluation from a practical standpoint, we discuss the application of our formulas in a Bayesian context, where prior distributions are assigned to the model parameters. Three existing algorithms for computing SCPDs are then described: a constrained alternating least squares (CALS) algorithm, a subspace-based solution and an algebraic solution for SCPDs with circulant factors. Subsequently, we present three numerical simulation scenarios, in which several specialized estimators based on these algorithms are proposed for concrete examples of SCPD involving circulant factors. In particular, the third scenario concerns the identification of a Wiener-Hammerstein system via the SCPD of an associated Volterra kernel. The statistical performance of the proposed estimators is assessed via Monte Carlo simulations, by comparing their Bayesian mean-square error with the expected CRB.
En savoir plus ### Classification of hyperspectral images by tensor modeling and additive morphological decomposition

spatial information, additive decomposition and tensor-PCA in classification of high dimensional images. In other words, in this scenario the influence of the classifier is limited and allow us to compare the different techniques of dimensionality reduction. To reliably evaluate the performance of the pro- posed method, the results were averaged over 25 different randomly selected training (of size five) for a number of feature yield by PCA and TPCA in the range of [1, . . . , 16]. For a best understanding of this comparison, the experiment considered in Fig. 7 illustrates the performance of ADL, AMD and DMP in both dimensional reduction approaches (PCA and TPCA). In the broader range of results, ADL exhibits a higher classification rate for the dimensionality size considered in this example. ADL led to the best classifi- cation results, as it can also be seen from Table 3. On the other hand, this experiment confirms our intuition that the inclusion of a spatial prior can sig- nificantly improve the classification results provided by using only spectral information. Fig. 8 shows the thematic classification maps for the pixel wise SVM and the spectral-spatial classification by morphological decomposition after the dimensional reduction step. Our approach involving morphological information is clearly better that its spectral equivalent. Additionally, ADL and tensor reduction has the best performance with more than 73% in overall classification in this very difficult scenario.
En savoir plus ### Performances estimation for tensor CP decomposition with structured factors

We consider a third-order tensor 1 Y ∈ R I 1 ×I 2 ×I 3 , following model (1) up to an additional noise tensor N with circular i.i.d. Gaussian entries of variance σ 2 : Y = X + N. (2) The factor matrices of X are denoted A (n) = [a (n) 1 , . . . , a (n) R ] ∈ R I n ×R , 1 ≤ n ≤ N = 3. The aim of this section is to derive the CRB of this model when some factors are structured. Throughout the rest of the paper, the equivalent vector model will be considered: y = vec( Y) = vec(X) + vec(N) = x(θ) + n, (3) where x(θ) = P R ### Tensor Decomposition Exploiting Structural Constraints for Brain Source Imaging

III. TENSOR-BASED BRAIN SOURCE LOCALIZATION In order to construct a tensor, we either need to apply a suit- able transformation, such as the wavelet transform or the short term Fourier transform, to the data or collect an additional dimension from the measurements. Subsequently, we consider EEG data containing interictal epileptic spike signals, which repeatedly occur in irregular time intervals, involving the same source regions at each occurrence. Following , , it is then possible to build a Space-Time-Spike (STS) tensor by stacking the spike-like signals observed at R different time instants along the third dimension of the tensor X . Assuming that the signals s p of the P sources are the same for each
En savoir plus ### Fast BEM solution for scattering problems using Quantized Tensor Train format

B. Quantization and QTT format Although some physical problems can naturally be formu- lated in terms of tensors and therefore solved by adapted techniques, a key point for the application of a tensor com- pression technique on integral equations is quantization , which enables a transition from a vector or a matrix to a tensor representation. For a vector x ∈ R I and I, d ∈ N such that ### Tensor decomposition for learning Gaussian mixtures from moments

Abstract In data processing and machine learning, an important challenge is to recover and exploit models that can represent accurately the data. We consider the problem of recovering Gaussian mix- ture models from datasets. We investigate symmetric tensor decomposition methods for tackling this problem, where the tensor is built from empirical moments of the data distribution. We consider identifiable tensors, which have a unique decomposition, showing that moment tensors built from spherical Gaussian mixtures have this property. We prove that symmetric tensors with interpolation degree strictly less than half their order are identifiable and we present an algorithm, based on simple linear algebra operations, to compute their decomposition. Illus- trative experimentations show the impact of the tensor decomposition method for recovering Gaussian mixtures, in comparison with other state-of-the-art approaches.
En savoir plus ### Atrial Signal Extraction in Atrial Fibrillation Electrocardiograms Using a Tensor Decomposition Approach

V. C ONCLUSIONS With the goal of overcoming the drawbacks of matrix- based BSS methods, this work has approached for the first time the problem of noninvasive AA extraction during AF from the perspective of tensor decompositions. Using a simple atrial signal model and a real AF ECG recording, numerical experiments demonstrate that BTD outperforms matrix-based methods in noisy and spatially-constrained sce- narios. Further research should aim at understanding the unexpected performance deterioration of the tensor tech- nique for short observation windows, designing automatic parameter selection methods for BTD, and evaluating its performance in more realistic AF signal models.
En savoir plus ### Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion

. However, this class of techniques destroy the HSI’s intrinsic structure when reconstructing the HR-HSI. In fact, since the original HSIs are organized in data cubes [38–40], repre- sentativeness and informativeness play an important role in real applications . For this reason, several tensor-based methods have been proposed and successfully applied. In , Dian et. al proposed a nonlocal sparse tensor factorization (NLSTF) method to fuse the LR-HSI and the HR-MSI. It regards the HSI as a three-order tensor, and decomposes the tensor into a core tensor multiplication by factor matrices representing the three dimensions. Yi et al.  proposed a weighted low-rank tensor recovery (WLRTR) model that treated the singular values differently. Similar to the non- local self-similarity across space (NSS) method , the nonlocal similarity between spectral-spatial cubic and spectral correlation can be characterized in tensors. Among all these effective tensor representation methods, parallel factor anal- ysis (PARAFAC) model, which is also called canonical polyadic (CP) decomposition [45–47], has been justified as a powerful method to decompose a tensor data onto a few rank- one tensors in a multilinear way. In HS data analysis, the CP decomposition has been widely used because it can describe the low-rank structure in HS data. Veganzones et al. used a novel compression-based nonnegative CP decomposition to the blind analysis of hyperspectral big data , and
En savoir plus ### Robust 3-way tensor decomposition and extended state Kalman filtering to extract fetal ECG

ABSTRACT This paper addresses the problem of fetal electrocardiogram (ECG) extraction from multichannel recordings. The pro- posed two-step method, which is applicable to as few as two channels, relies on (i) a deterministic tensor decomposition approach, (ii) a Kalman ﬁltering. Tensor decomposition cri- teria that are robust to outliers are proposed and used to better track weak traces of the fetal ECG. Then, the state parameters used within an extended realistic nonlinear dynamic model for extraction of N ECGs from M mixtures of several ECGs and noise are estimated from the loading matrices provided by the ﬁrst step. Application of the proposed method on actual data shows its signiﬁcantly superior performance in compari- son to the classic methods.
En savoir plus ### A Tensor Decomposition Approach to Noninvasive Atrial Activity Extraction in Atrial Fibrillation ECG

An additional experiment was carried out by changing the sample size of the ECG recording, estimating the f-wave and calculating its SC, as shown in Fig. 4. The parameter L was computed as in the previous experiment and the number of block terms was R = 3 as well. These experiments indi- cate that the f-wave can be approximated as a sum of com- plex exponentials and that BTD is robust to temporal and spa- tial constrained scenarios, presenting better SC index over the benchmark methods. It is important to note, however, that a successful tensor decomposition by BTD strongly depends on the proper initialization of its factors. Devising an appropriate initialization for BTD remains a challenge.
En savoir plus