Publications

2022

• Unifying conditional and unconditional semantic image synthesis with OCO-GAN
  
  • Careil Marlène
  • Lathuilière Stéphane
  • Couprie Camille
  • Verbeek Jakob
  , 2022. Generative image models have been extensively studied in recent years. In the unconditional setting, they model the marginal distribution from unlabelled images. To allow for more control, image synthesis can be conditioned on semantic segmentation maps that instruct the generator the position of objects in the image. While these two tasks are intimately related, they are generally studied in isolation. We propose OCO-GAN, for Optionally COnditioned GAN, which addresses both tasks in a unified manner, with a shared image synthesis network that can be conditioned either on semantic maps or directly on latents. Trained adversarially in an end-to-end approach with a shared discriminator, we are able to leverage the synergy between both tasks. We experiment with Cityscapes, COCO-Stuff, ADE20K datasets in a limited data, semi-supervised and full data regime and obtain excellent performance, improving over existing hybrid models that can generate both with and without conditioning in all settings. Moreover, our results are competitive or better than state-of-the art specialised unconditional and conditional models.
• Approximate Bayesian computation with the sliced-Wasserstein distance
  
  • Nadjahi Kimia
  • de Bortoli Valentin
  • Durmus Alain
  • Badeau Roland
  • Şimşekli Umut
  , 2022. Approximate Bayesian Computation (ABC) is a popular method for approximate inference in generative models with intractable but easy-to-sample likelihood. It constructs an approximate posterior distribution by finding parameters for which the simulated data are close to the observations in terms of summary statistics. These statistics are defined beforehand and might induce a loss of information, which has been shown to deteriorate the quality of the approximation. To overcome this problem, Wasserstein-ABC has been recently proposed, and compares the datasets via the Wasserstein distance between their empirical distributions, but does not scale well to the dimension or the number of samples. We propose a new ABC technique, called Sliced-Wasserstein ABC and based on the Sliced-Wasserstein distance, which has better computational and statistical properties. We derive two theoretical results showing the asymptotical consistency of our approach, and we illustrate its advantages on synthetic data and an image denoising task. (10.1109/icassp40776.2020.9054735)
  DOI : 10.1109/icassp40776.2020.9054735
• Custom Structure Preservation in Face Aging
  
  • Gomez-Trenado Guillermo
  • Lathuilière Stéphane
  • Mesejo Pablo
  • Cordón Óscar
  , 2022. In this work, we propose a novel architecture for face age editing that can produce structural modifications while maintaining relevant details present in the original image. We disentangle the style and content of the input image and propose a new decoder network that adopts a style-based strategy to combine the style and content representations of the input image while conditioning the output on the target age. We go beyond existing aging methods allowing users to adjust the degree of structure preservation in the input image during inference. To this purpose, we introduce a masking mechanism, the CUstom Structure Preservation module, that distinguishes relevant regions in the input image from those that should be discarded. CUSP requires no additional supervision. Finally, our quantitative and qualitative analysis which include a user study, show that our method outperforms prior art and demonstrates the effectiveness of our strategy regarding image editing and adjustable structure preservation. Code and pretrained models are available at https://github.com/guillermogotre/CUSP.
• Dominating, Locating-Dominating and Identifying Codes in the q-ary Lee Hypercube
  
  • Hudry Olivier
  • Charon Irene
  • Lobstein Antoine
  , 2022.
• Negative Sampling Strategies for Contrastive Self-Supervised Learning of Graph Representations
  
  • Hafidi Hakim
  • Ghogho Mounir
  • Ciblat Philippe
  • Swami Ananthram
  Signal Processing, Elsevier, 2022, 190 (4). Contrastive learning has become a successful approach for learning powerful text and image representations in a self-supervised manner. Contrastive frameworks learn to distinguish between representations coming from augmentations of the same data point (positive pairs) and those of other (negative) examples. Recent studies aim at extending methods from contrastive learning to graph data. In this work, we propose a general framework for learning node representations in a self supervised manner called Graph Constrastive Learning (GraphCL). It learns node embeddings by maximizing the similarity between the nodes representations of two randomly perturbed versions of the same graph. We use graph neural networks to produce two representations of the same node and leverage a contrastive learning loss to maximize agreement between them. We investigate different standard and new negative sampling strategies as well as a comparison without negative sampling approach. We demonstrate that our approach significantly outperforms the state-of-the-art in unsupervised learning on a number of node classification benchmarks in both transductive and inductive learning setups.
• A general sample complexity analysis of vanilla policy gradient
  
  • Yuan Rui
  • Gower Robert M
  • Lazaric Alessandro
  , 2022. We adapt recent tools developed for the analysis of Stochastic Gradient Descent (SGD) in non-convex optimization to obtain convergence and sample complexity guarantees for the vanilla policy gradient (PG). Our only assumptions are that the expected return is smooth w.r.t. the policy parameters, that its H-step truncated gradient is close to the exact gradient, and a certain ABC assumption. This assumption requires the second moment of the estimated gradient to be bounded by A ≥ 0 times the suboptimality gap, B ≥ 0 times the norm of the full batch gradient and an additive constant C ≥ 0, or any combination of aforementioned. We show that the ABC assumption is more general than the commonly used assumptions on the policy space to prove convergence to a stationary point. We provide a single convergence theorem that recovers the O(−4) sample complexity of PG. Our results also affords greater flexibility in the choice of hyper parameters such as the step size and places no restriction on the batch size m, including the single trajectory case (i.e., m = 1). We then instantiate our theorem in different settings, where we both recover existing results and obtained improved sample complexity, e.g., for convergence to the global optimum for Fisher-nondegenerated parameterized policies.
• FAST STRATEGIES FOR MULTI-TEMPORAL SPECKLE REDUCTION OF SENTINEL-1 GRD IMAGES
  
  • Meraoumia Inès
  • Dalsasso Emanuele
  • Denis Loïc
  • Tupin Florence
  , 2022. Reducing speckle and limiting the variations of the physical parameters in Synthetic Aperture Radar (SAR) images is often a key-step to fully exploit the potential of such data. Nowadays, deep learning approaches produce state of the art results in single-image SAR restoration. Nevertheless, huge multi-temporal stacks are now often available and could be efficiently exploited to further improve image quality. This paper explores two fast strategies employing a singleimage despeckling algorithm, namely SAR2SAR [1], in a multi-temporal framework. The first one is based on Quegan filter [2] and replaces the local reflectivity pre-estimation by SAR2SAR. The second one uses SAR2SAR to suppress speckle from a ratio image encoding the multi-temporal information under the form of a "super-image", i.e. the temporal arithmetic mean of a time series. Experimental results on Sentinel-1 GRD data show that these two multi-temporal strategies provide improved filtering results while adding a limited computational cost. (10.1109/IGARSS46834.2022.9883448)
  DOI : 10.1109/IGARSS46834.2022.9883448
• A Review of Machine Learning Techniques in Analog Integrated Circuit Design Automation
  
  • Mina Rayan
  • Jabbour Chadi
  • Sakr George E
  Electronics, MDPI, 2022, 11 (3), pp.435. Analog integrated circuit design is widely considered a time-consuming task due to the acute dependence of analog performance on the transistors’ and passives’ dimensions. An important research effort has been conducted in the past decade to reduce the front-end design cycles of analog circuits by means of various automation approaches. On the other hand, the significant progress in high-performance computing hardware has made machine learning an attractive and accessible solution for everyone. The objectives of this paper were: (1) to provide a comprehensive overview of the existing state-of-the-art machine learning techniques used in analog circuit sizing and analyze their effectiveness in achieving the desired goals; (2) to point out the remaining open challenges, as well as the most relevant research directions to be explored. Finally, the different analog circuits on which machine learning techniques were applied are also presented and their results discussed from a circuit designer perspective. (10.3390/electronics11030435)
  DOI : 10.3390/electronics11030435
• Curves on Frobenius classical surfaces in $\mathbb{P}^{3}$ over finite fields
  
  • Berardini Elena
  • Nardi Jade
  Acta Arithmetica, Instytut Matematyczny PAN, 2022, 205 (4), pp.323-340. (10.4064/aa211118-12-9)
  DOI : 10.4064/aa211118-12-9
• Codes in the q-ary Lee Hypercube
  
  • Hudry Olivier
  • Charon Irène
  • Lobstein Antoine
  WSEAS Transactions on Mathematics, World Scientific and Engineering Academy and Society (WSEAS), 2022, 21, pp.173-186. (10.37394/23206.2022.21.24)
  DOI : 10.37394/23206.2022.21.24
• SELF ATTENTION DEEP GRAPH CNN CLASSIFICATION OF TIMES SERIES IMAGES FOR LAND COVER MONITORING
  
  • Chaabane Ferdaous
  • Réjichi Safa
  • Tupin Florence
  , 2022. Time Series of Satellite Imagery (SITS) acquired by recent Earth observation systems represent an important source of information that supports several remote sensing applications related to monitoring the dynamics of the Earth's surface over large areas. A major challenge then is to design new deep learning models that can take into account intelligently the complementarity between temporal and spatial contexts that characterize these data structures. In this work, we propose to use an adapted self-attention convolutional neural network for spatio-temporal graphs classification that exploits both spatial and temporal dimensions. The graphs will be generated from a series of temporal images that are segmented into different regions. Those graphs are then classified using the Self-Attention Deep Graph CNN (DGCNN) model to highlight the temporal evolution of land cover areas through the construction of a spatio-temporal Map.
• What are the best systems? New perspectives on NLP Benchmarking
  
  • Irurozki Ekhine
  • Colombo Pierre
  • Noiry Nathan
  • Clémençon Stéphan
  , 2022. In Machine Learning, a benchmark refers to an ensemble of datasets associated with one or multiple metrics together with a way to aggregate different systems performances. They are instrumental in (i) assessing the progress of new methods along different axes and (ii) selecting the best systems for practical use. This is particularly the case for NLP with the development of large pre-trained models (e.g. GPT, BERT) that are expected to generalize well on a variety of tasks. While the community mainly focused on developing new datasets and metrics, there has been little interest in the aggregation procedure, which is often reduced to a simple average over various performance measures. However, this procedure can be problematic when the metrics are on a different scale, which may lead to spurious conclusions. This paper proposes a new procedure to rank systems based on their performance across different tasks. Motivated by the social choice theory, the final system ordering is obtained through aggregating the rankings induced by each task and is theoretically grounded. We conduct extensive numerical experiments (on over 270k scores) to assess the soundness of our approach both on synthetic and real scores (e.g. GLUE, EXTREM, SEVAL, TAC, FLICKR). In particular, we show that our method yields different conclusions on state-of-the-art systems than the mean-aggregation procedure while being both more reliable and robust
• Comparing Deep Models and Evaluation Strategies for Multi-Pitch Estimation in Music Recordings
  
  • Weis Christof
  • Peeters Geoffroy
  IEEE/ACM Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, 2022, 30, pp.2814-2827. (10.1109/TASLP.2022.3200547)
  DOI : 10.1109/TASLP.2022.3200547
• Information Leakage in Code-based Masking: A Systematic Evaluation by Higher-Order Attacks
  
  • Cheng Wei
  • Guilley Sylvain
  • Danger Jean-Luc
  IEEE Transactions on Information Forensics and Security, Institute of Electrical and Electronics Engineers, 2022, 17, pp.1624-1638. Code-based masking is a recent line of research on masking schemes aiming at provably counteracting side-channel attacks. It generalizes and unifies many masking schemes within a coding-theoretic formalization. In code-based masking schemes, the tuning parameters are the underlying linear codes, whose choice significantly affects the side-channel resilience. In this paper, we investigate the exploitability of the information leakage in code-based masking and present attack-based evaluation results of higher-order optimal distinguisher (HOOD). Particularly, we consider two representative instances of code-based masking, namely inner product masking (IPM) and Shamir’s secret sharing (SSS) based masking. Our results do confirm the state-of-the-art theoretical derivatives in an empirical manner with numerically simulated measurements. Specifically, theoretical results are based on quantifying information leakage; we further complete the panorama with attack-based evaluations by investigating the exploitability of the leakage. Moreover, we classify all possible candidates of linear codes in IPM with 2 and 3 shares and (3, 1)-SSS based masking, and highlight both optimal and worst codes for them. Relying on our empirical evaluations, we therefore recommend investigating the coding-theoretic properties to find the best linear codes in strengthening instances of code-based masking. As for applications, our attack-based evaluation directly empowers designers, by employing optimal linear codes, to enhance the protection of code-based masking. Our framework leverages simulated leakage traces, hence allowing for source code validation or patching in case it is found to be attackable. (10.1109/TIFS.2022.3167914)
  DOI : 10.1109/TIFS.2022.3167914
• Delaunay Painting: Perceptual image coloring from raster contours with gaps
  
  • Parakkat Amal Dev
  • Memari Pooran
  • Cani Marie-Paule
  Computer Graphics Forum, Wiley, 2022. We introduce Delaunay Painting, a novel and easy-to-use method to flat-color contour-sketches with gaps. Starting from a Delaunay triangulation of the input contours, triangles are iteratively filled with the appropriate colors, thanks to the dynamic update of flow values calculated from color hints. Aesthetic finish is then achieved, through energy minimisation of contour curves and further heuristics enforcing the appropriate sharp corners. To be more efficient, the user can also make use of our color diffusion framework which automatically extends coloring to small, internal regions such as those delimited by hatches. The resulting method robustly handles input contours with strong gaps. As an interactive tool, it minimizes user's efforts and enables any coloring strategy, as the result does not depend on the order of interactions. We also provide an automatized version of the coloring strategy for quick segmentation of contours images, that we illustrate with an application to medical imaging.
• M A T R I X Transpositions. Spécial Prod : de l’autre côté du miroir
  
  • Zayana Karim
  • Boyer Ivan
  • Massard Marc-Aurèle
  • Rabiet Victor
  CultureMath, ENS, 2022.
• Conditional independence testing via weighted partial copulas and nearest neighbors
  
  • Bianchi Pascal
  • Elgui Kevin
  • Portier François
  Journal of Multivariate Analysis, Elsevier, 2022, 30 (3), pp.1117-1147. This paper introduces the \textit{weighted partial copula} function for testing conditional independence. The proposed test procedure results from these two ingredients: (i) the test statistic is an explicit Cramer-von Mises transformation of the \textit{weighted partial copula}, (ii) the regions of rejection are computed using a bootstrap procedure which mimics conditional independence by generating samples from the product measure of the estimated conditional marginals. Under conditional independence, the weak convergence of the \textit{weighted partial copula proces}s is established when the marginals are estimated using a smoothed local linear estimator. Finally, an experimental section demonstrates that the proposed test has competitive power compared to recent state-of-the-art methods such as kernel-based test.
• As if by magic: self-supervised training of deep despeckling networks with MERLIN
  
  • Dalsasso Emanuele
  • Denis Loïc
  • Tupin Florence
  IEEE Transactions on Geoscience and Remote Sensing, Institute of Electrical and Electronics Engineers, 2022, 60, pp.1-13. Speckle fluctuations seriously limit the interpretability of synthetic aperture radar (SAR) images. Speckle reduction has thus been the subject of numerous works spanning at least four decades. Techniques based on deep neural networks have recently achieved a new level of performance in terms of SAR image restoration quality. Beyond the design of suitable network architectures or the selection of adequate loss functions, the construction of training sets is of uttermost importance. So far, most approaches have considered a supervised training strategy: the networks are trained to produce outputs as close as possible to speckle-free reference images. Speckle-free images are generally not available, which requires resorting to natural or optical images or the selection of stable areas in long time series to circumvent the lack of ground truth. Self-supervision, on the other hand, avoids the use of speckle-free images. We introduce a self-supervised strategy based on the separation of the real and imaginary parts of single-look complex (SLC) SAR images, called coMplex sElf-supeRvised despeckLINg (MERLIN), and show that it offers a straightforward way to train all kinds of deep despeckling networks. Networks trained with MERLIN take into account the spatial correlations due to the SAR transfer function specific to a given sensor and imaging mode. By requiring only a single image, and possibly exploiting large archives, MERLIN opens the door to hassle-free as well as large-scale training of despeckling networks. The code of the trained models is made freely available at https://gitlab.telecom-paris.fr/RING/MERLIN . (10.1109/TGRS.2021.3128621)
  DOI : 10.1109/TGRS.2021.3128621
• Tunnel Magnetoresistance Based Passive Resistance Replacement in Hybrid MTJ-CMOS Integration
  
  • Wu Yu-Ang
  • Xie Xinshu
  • Tong Xinfang
  • Di Yantong
  • Naviner Lirida
  • Liu Bo
  • Xiao Jian
  • Cai Hao
  IEEE Transactions on Nanotechnology, Institute of Electrical and Electronics Engineers, 2022, 21, pp.638-647. Previous theoretical and experimental works revealed the novel factors that Magnetic tunnel junction (MTJ) can be integrated into novel hybrid circuits except for memory applications. This paper makes exploitation of tunnel magnetoresistacne based replacement in diminishing layout penalty of on-chip passive component during circuit design and takes sigma-delta analog-to-digital converter (SD-ADC), resistor-based temperature sensor (-55 ∘ C ∼125∘ C) as two case study where large resistance is needed and restricts the scaling down. Considering the application in MRAM, the mainstream field of MTJ process and dealing with the problems of MRAM in wide temperature write operation, two temperature adaptive write schemes of MRAM are also proposed as the further applications of the proposed MTJ-based temperature sensor. The research of these circuits covers major characteristics of MTJ as the passive component, including area, variation and temperature characteristics. Large CMOS resistance in SD-ADC and bridge transducer in resistor-based temperature sensor are replaced by MTJ-based resistors. Simulation results reveal that the layout area of passive resistors in resistor-capacitor (RC) integrator was greatly reduced by 94.52% in comparison with fully 28nm CMOS design or 94.13% for wide temperature use when other performance is almost unchanged. In addition, the MTJ based bridge transducer in resistor-based temperature sensor can reduce the resistance layout area by over 90% with better linearity comparing with general CMOS resistor-based temperature sensor designs. Based on the MTJ-based temperature sensor, the two different adaptive write circuits help reduce write power consumption and delay of MRAM respectively for wide temperature use. (10.1109/TNANO.2022.3216778)
  DOI : 10.1109/TNANO.2022.3216778
• On the Complexity of Determining Whether there is a Unique Hamiltonian Cycle or Path
  
  • Hudry Olivier
  • Lobstein Antoine
  WSEAS Transactions on Mathematics, World Scientific and Engineering Academy and Society (WSEAS), 2022, 21, pp.433-446. The decision problems of the existence of a Hamiltonian cycle or of a Hamiltonian path in a given graph, and of the existence of a truth assignment satisfying a given Boolean formula C, are well-known NPcomplete problems. Here we study the problems of the uniqueness of a Hamiltonian cycle or path in an undirected, directed or oriented graph, and show that they have the same complexity, up to polynomials, as the problem U-SAT of the uniqueness of an assignment satisfying C. As a consequence, these Hamiltonian problems are NP-hard and belong to the class DP, like U-SAT. (10.37394/23206.2022.21.51)
  DOI : 10.37394/23206.2022.21.51
• Online Unsupervised Domain Adaptation for Person Re-identification
  
  • Rami Hamza
  • Ospici Matthieu
  • Lathuilière Stéphane
  , 2022. Unsupervised domain adaptation for person re-identification (Person Re-ID) is the task of transferring the learned knowledge on the labeled source domain to the unlabeled target domain. Most of the recent papers that address this problem adopt an offline training setting. More precisely, the training of the Re-ID model is done assuming that we have access to the complete training target domain data set. In this paper, we argue that the target domain generally consists of a stream of data in a practical real-world application, where data is continuously increasing from the different network's cameras. The Re-ID solutions are also constrained by confidentiality regulations stating that the collected data can be stored for only a limited period, hence the model can no longer get access to previously seen target images. Therefore, we present a new yet practical online setting for Unsupervised Domain Adaptation for person Re-ID with two main constraints: Online Adaptation and Privacy Protection. We then adapt and evaluate the state-of-the-art UDA algorithms on this new online setting using the well-known Market-1501, Duke, and MSMT17 benchmarks.
• Playable Environments: Video Manipulation in Space and Time
  
  • Menapace Willi
  • Lathuilière Stéphane
  • Siarohin Aliaksandr
  • Theobalt Christian
  • Tulyakov Sergey
  • Golyanik Vladislav
  • Ricci Elisa
  , 2021, 97, pp.116366. We present Playable Environments - a new representation for interactive video generation and manipulation in space and time. With a single image at inference time, our novel framework allows the user to move objects in 3D while generating a video by providing a sequence of desired actions. The actions are learnt in an unsupervised manner. The camera can be controlled to get the desired viewpoint. Our method builds an environment state for each frame, which can be manipulated by our proposed action module and decoded back to the image space with volumetric rendering. To support diverse appearances of objects, we extend neural radiance fields with style-based modulation. Our method trains on a collection of various monocular videos requiring only the estimated camera parameters and 2D object locations. To set a challenging benchmark, we introduce two large scale video datasets with significant camera movements. As evidenced by our experiments, playable environments enable several creative applications not attainable by prior video synthesis works, including playable 3D video generation, stylization and manipulation. Further details, code and examples are available at https://willi-menapace.github.io/playable-environments-website
• Vector-Valued Least-Squares Regression under Output Regularity Assumptions
  
  • Brogat-Motte Luc
  • Rudi Alessandro
  • Brouard Celine
  • Rousu Juho
  • d'Alché-Buc Florence
  Journal of Machine Learning Research, Microtome Publishing, 2022. We propose and analyse a reduced-rank method for solving least-squares regression problems with infinite dimensional output. We derive learning bounds for our method, and study under which setting statistical performance is improved in comparison to full-rank method. Our analysis extends the interest of reduced-rank regression beyond the standard low-rank setting to more general output regularity assumptions. We illustrate our theoretical insights on synthetic least-squares problems. Then, we propose a surrogate structured prediction method derived from this reduced-rank method. We assess its benefits on three different problems: image reconstruction, multi-label classification, and metabolite identification.
• Safety, Security and Performance Assessment of Security Countermeasures with SysML-Sec
  
  • Sultan Bastien
  • Apvrille Ludovic
  • Jaillon Philippe
  , 2022. Deploying security countermeasures on Cyber-Physical Systems (CPS) can induce side-effects that can exceed their benefits. When CPS are safety-critical systems, performing efficiency and impact assessments of security countermeasures early in the design flow is essential. The paper introduces the W-Sec method, based on SysML-Sec. The W-Sec method consists in two interwoven formal modeling and verification cycles aiming at providing countermeasures with objective and quantitative efficiency and impact assessments in terms of safety, security and performance. The paper evaluates the W-Sec method with an autonomous rover swarm case-study, and finally discusses the method's strengths and weaknesses highlighted by the case-study results. (10.5220/0010832300003119)
  DOI : 10.5220/0010832300003119
• Unipolar quantum technology enabling high-speed free-space communication in the long-wave infrared regime
  
  • Didier P.
  • Dely H.
  • Spitz O.
  • Awwad Elie
  • Bonazzi Thomas
  • Rodriguez Etienne
  • Sirtori C.
  • Grillot Frédéric
  , 2022. <p>A combination of unipolar quantum laser, modulator and detector allows us to demonstrate a free-space communication at 40 Gbits/s and 9 µm far-infrared wavelength. The distance between the emitter and the receiver is 31 meters.</p> (10.1364/CLEO_AT.2022.JTh6A.5)
  DOI : 10.1364/CLEO_AT.2022.JTh6A.5