The Mathematical Sciences Colloquium series is held each semester, generally on Mondays at 4pm, and is sponsored by the math department. Faculty in the math department invite speakers from all areas of mathematics, and the talks are open to all members of the RPI community. The calendar is organized by the colloquium chair Yangyang Xu.

2020

Nov
16
2020
https://rensselaer.webex.com/meet/xuy21 4:30 pm

Apr
6
2020

Mar
30
2020

2019

Nov
4
2019
Amos Eaton 216 4:00 pm
View event flyer

Apr
8
2019
Neural networks as interacting particle systems: understanding global convergence of parameter optimization dynamics

Abstract:   The performance of neural networks on high-dimensional data distributions suggests that it may be possible to parameterize a representation of a target high-dimensional function with controllably small errors, potentially outperforming standard interpolation methods. We demonstrate, both theoretically and numerically, that this is indeed the case. We map the parameters of a neural network to a system of particles relaxing with an interaction potential determined by the loss function. This mapping gives rise to a deterministic partial differential equation that governs the parameter evolution under gradient descent dynamics. We also show that in the limit that the number of parameters n is large, the landscape of the mean-squared error becomes convex and the representation error in the function scales link n^{-1}. In this limit, we prove a dynamical variant of the universal approximation theorem showing that the optimal representation can be attained by stochastic gradient descent, the algorithm ubiquitously used for parameter optimization in machine learning. This conceptual framework can be leveraged to develop algorithms that accelerate optimization using non-local transport. I will conclude by showing that using neuron birth/death processes in parameter optimization guarantees global convergence and provides a substantial acceleration in practice.

Amos Eaton 214 4:00 pm
View event flyer

Mar
25
2019
Deep Learning with Graph Structured Data: Methods, Theory, and Applications

Abstract: Graphs are universal representations of pairwise relationship. A trending topic in deep learning is to extend the remarkable success of well-established neural network architectures (e.g., CNN and RNN) for Euclidean structured data to irregular domains, including notably, graphs. A proliferation of graph neural networks (e.g., GCN) emerged recently, but the scalability challenge for training and inference persists. The essence of the problem is the prohibitive computational cost of computing a mini-batch, owing to the recursive expansion of neighborhoods. We propose a scalable approach, coined FastGCN, based on neighborhood sampling to reduce the mini-batch computation. FastGCN achieves orders of magnitude improvement in training time, compared with a standard implementation of GCN. Predictions remain comparably accurate. A curious question for this approach is why stochastic gradient descent (SGD) training ever converges. In the second part of this talk, we analyze that the gradient estimator so computed is not unbiased but consistent. We thus extend the standard SGD results for unbiased gradients to consistent gradients and show that their convergence behaviors are similar. These results are important and may spawn new interest in the machine learning community, since in many learning scenarios unbiased estimators may not be efficient to compute, and hence other nonstandard but fast gradient estimators serve as sound alternatives.

Amos Eaton 214 4:00 pm
View event flyer

2018

2017

Nov
13
2017
Ford Foundation Professor in the Department of Mathematical Sciences and the founding director of IPRPI, the Inverse Problems Center at Rensselaer Polytechnic Institute
Chapel and Cultural Center, 2125 Burdett Ave, Troy, NY. Joyce’s family will receive friends and colleagues beginning at 9:15 am prior to the service. 10:00 am

May
1
2017

Apr
24
2017
"Untangling cause and effect in active neuronal dendrites"
Amos Eaton 214, Time: 4:00 pm

Mar
7
2017
A Roadmap to Well Posed and Stable Problems in Computational Physics

 

All numerical calculations will fail to provide a reliable answer unless the continuous problem under consideration is well posed. Well-posedness depends in most cases only on the choice of boundary conditions. In this paper we will highlight this fact, and exemplify by discussing well-posedness of a prototype problem: the time-dependent compressible Navier–Stokes equations. We do not deal with discontinuous problems, smooth solutions with smooth and compatible data are considered. 

In particular, we will discuss how many boundary conditions are required, where to impose them and which form they should have in order to obtain a well posed problem. Once the boundary conditions are known, one issue remains; they can be imposed weakly or strongly. It is shown that the weak and strong boundary procedures produce similar continuous energy estimates. We conclude by relating the well-posedness results to energy-stability of a numerical approximation on summation-by-parts form. It is shown that the results obtained for weak boundary conditions in the well-posedness analysis lead directly to corresponding stability results for the discrete problem, if schemes on summation-by-parts form and weak boundary conditions are used.

The analysis in this paper is general and can without difficulty be extended to any coupled system of partial differential equations posed as an initial boundary value problem coupled with a numerical method on summation-by parts form with weak boundary conditions. Our ambition in this paper is to give a general roadmap for how to construct a well posed continuous problem and a stable numerical approximation, not to give exact answers to specific problems.

 

Sage 3510, Time: 4:00 pm
View event flyer

2016

Dec
12
2016
Data analysis - from oscillatory patterns to geometric structures

There are numerous and diverse challenges associated with analyzing data collected from different fields of science and engineering. This talk consists of two parts.  First, time-dependent oscillatory signals occur in a wide range of fields, including geophysics, biology, medicine, finance and social dynamics. Of great interest are techniques that decompose the time-dependent signals into multiple oscillatory components, with time-varying amplitudes and instantaneous frequencies. Such decompositions can help us better describe and quantify the underlying dynamics that govern the system. I will present a new advance in time-frequency representations whose effectiveness is justified by both numerical experiments and theoretical analysis.  Second, the high-dimensionality of point cloud data makes investigating such data difficult. Fortunately, these data often locally concentrate along a low-dimensional subspace and this makes the problem more tractable. I will talk about utilizing low-dimensional structures for various data analysis objectives, ranging from recovering the underlying data in the presence of complex noise including Gaussian additive noise and large sparse corruptions, to recognizing subspace-based patterns in data, from robust algorithm design to theoretical analysis. The techniques for learning subspaces have broad applications: image processing, computer vision, bioinformatics, medicine, etc.  At the end, I will talk about some future directions where both fields are involved.

Yi (Grace) Wang, Syracuse University
Lally 104; Time: 4:00 pm
View event flyer

Dec
1
2016
Dane Taylor, University of North Carolina
Lally 104; Time: 4:00 pm
View event flyer

Nov
7
2016
Helen Moore
Lally 104, Time: 4:00 pm
View event flyer

Oct
31
2016

Oct
27
2016
When black holes collide: a new window on the universe

LIGO’s detection of gravitational waves from a binary black hole merger inaugurates a completely new mode of observational astronomy and represents the culmination of a quest lasting half a century. After a brief review of gravitational waves in general relativity, I will discuss the detection itself. How do the LIGO instruments work? How do we know the signal was caused by a binary black hole merger? What does this detection tell us about binary black holes? Then I will focus on how this moment came to pass. The detection required many ingredients to be in place including (1) developments in theoretical relativity to allow proof that gravitational waves were not coordinate artifacts; (2) a bold vision to recognize that gravitational wave detection was not impossible; (3) technological developments of novel vacuum systems, lasers, optical coatings, active seismic isolation, etc.; (4) the successful conclusion of a 35 year effort to simulate binary black holes on the computer; (5) development of sophisticated, new data analysis methods to tease a waveform from noisy data; (5) the growth of the field of gravitational wave science from a handful of practitioners to the more than 1000 authors on the detection paper; and finally (6) the (nearly) unwavering support of the National Science Foundation. The first detection was followed by a second one in this first "science run" and soon another science run will begin. I will end with discussion of the future — more binary black holes, other sources of gravitational waves and what we might learn, instrument upgrades, new facilities — and other ways to detect gravitational waves — from space and from monitoring millisecond pulsars.

Beverly K. Berger
Lally 104, Time: 4:00 pm
View event flyer

May
5
2016
Sparse and multilevel methods for particle-in-cell simulations in plasma physics

The "particle-in-cell" (PIC) method is a technique for solving kinetic PDEs that has been a standard simulation tool in plasma physics for 50 years. Originally, the method was an attempt to circumvent the curse of dimensionality when solving high-dimensional kinetic PDEs by combining particle- and grid-based representations. The technique has been enormously successful in many regards but even today, generating a quantitatively accurate solution in complex, three-dimensional geometry requires many hours on a massively parallel machine. Two prominent reasons for the massive complexity of PIC schemes are the statistical noise introduced by the particle representation and the fact that multiple disparate physical time-scales necessitate taking enormous numbers of time-steps. We present approaches to circumventing each of these difficulties. First, we propose the use of 'sparse grids' (see e.g. Griebel et al, 1990) to estimate grid-based quantities from particle information. We show that this can dramatically reduce statistical errors while only increasing grid-based error by a logarithmic factor. Second, we present a multilevel - in time - technique in the spirit of the multilevel Monte Carlo (MLMC) method (see e.g. Giles, 2008). The idea is to combine information from simulations using many particles and a large time step on the one hand with simulations using few particles and a small time step on the other. This is done in such a way as to generate a new solution that mimics one with many particles and a small time-step, but at dramatically reduced cost. Scalings of the computational complexity of PIC codes using each of these approaches will be discussed, and proof-of-principle results will be presented from solving the 4-D Vlasov-Poisson PDE. Finally, we will discuss the prospects for combining the two approaches, parallel issues, and other future directions.

JROWL 2C14, Time: 4:00 pm
View event flyer

Apr
18
2016
Synchronization Problems and the Diffusion Geometry of Shape Spaces

Kernel-based non-linear dimensionality reduction methods, such as Local Linear Embedding (LLE) and Laplacian Eigenmaps, rely heavily upon pairwise distances or similarity scores, with which one can construct and study a weighted graph associated with the data set. When each individual data object carries structural details, the correspondence relations between these structures provide additional information that can be leveraged for studying the data set using the graph. In this talk, I will introduce the framework of Horizontal Diffusion Maps (HDM), a generalization of Diffusion Maps in manifold learning. This framework models a data set with pairwise structural correspondences as a fibre bundle equipped with a connection. We further demonstrate the advantage of incorporating such additional information and study the asymptotic behavior of HDM on general fibre bundles. In a broader context, HDM reveals the sub-Riemannian structure of high-dimensional data sets, and provides a nonparametric learning framework for data sets with structural correspondences. Mre generally, it can be viewed as geometric realization of synchronization problems. A synchronization problem for a group $G$ and a graph $\Gamma=\left(V, E\right)$ searches for an assignment of elements in $G$ to edges of $\Gamma$ so the overall configuration minimizes an energy functional under certain compatibility constraints; it is essentially a generalization to the non-commutative setting of the little Grothendieck problem. In this talk, I will also explain some recent work on the cohomological nature of this type of problems. Our interest in synchronization and diffusion geometry arises from the emerging field of automated geometric morphometrics. At present, evolutionary anthropologist using physical traits to study evolutionary relationships among living and extinct animals analyze morphological data extracted from carefully defined anatomical landmarks. Identifying and recording these landmarksis time consuming and can be done accurately only by trained morphometricians. This necessity renders these studies inaccessible to non-morphologists and causes phenomics to lag behind genomics in elucidating evolutionary patterns. This talk will also cover the application of our work to the automation of this morphological analysis in a landmark-free manner.

Amos Eaton 214, Time: 4:00 pm
View event flyer

Apr
11
2016
Initial Value Problems and Initial-Boundary Value Problems for Nonlinear Evolution Equations

Nonlinear evolution PDEs are a central topic in mathematical research, not only due to their inner beauty and complexity but also thanks to their broad range of real-world applications, from physics and biology to finance and economics. The first part of this talk is devoted to a new approach develop- ed in collaboration with A.S. Fokas and A. Himonas for the well-posedness of initial-boundary value problems for such PDEs in one spatial dimension. In particular, it is shown that the nonlinear Schrödinger (NLS) and the Korteweg-de-Vries (KdV) equations are well-posed on the half-line with data in appropriate Sobolev spaces. The second part of the talk is concerned with the initial value problem for a nonlocal, nonlinear evolution PDE of Camassa- Holm type with cubic nonlinearity, which is integrable, admits periodic and non-periodic multi-peakon traveling wave solutions, and can be derived as a shallow water approximation to the celebrated Euler equations. Finally, the third part of the talk addresses a long-standing open question, namely the nonlinear stage of modulational instability (a.k.a. Benjamin-Feir instability), which is one of the most ubiquitous phenomena in nonlinear science. For all those physical systems governed by the focusing NLS equation, a precise characterization of the nonlinear stage of modulational instability is obtained by computing explicitly the long-time asymptotic behavior of the relevant initial value problem formulated with nonzero boundary conditions at infinity.”

Amos Eaton 214, Time: 4:00 pm
View event flyer

2015