Colloquia, Seminars, & Graduate Student Poster Sessions

Artificial intelligence (AI) has become woven into therapeutic discovery to accelerate drug discovery and development processes since the emergence of deep learning. For drug discovery, the goal is to identify drug molecules with desirable pharmaceutical properties. I will discuss our deep generative models that relax the discrete molecule space into a differentiable one and reformulate the combinatorial optimization problem into a differentiable optimization problem, which can be solved efficiently. On the other hand, drug development focuses on conducting clinical trials to evaluate the safety and effectiveness of the drug on human bodies. To predict clinical trial outcomes, I design deep representation learning methods to capture the interaction between multi-modal clinical trial features (e.g., drug molecules, patient information, disease information), which achieves 0.847 F1 score in predicting phase III approval. Finally, I will present my future works in geometric deep learning for drug discovery and predictive model for drug development.

Bio: Tianfan Fu is a Ph.D. candidate in the School of Computational Science and Engineering at the Georgia Institute of Technology, advised by Prof. Jimeng Sun. His research interest lies in machine learning for drug discovery and development. Particularly, he is interested in generative models on both small-molecule & macro-molecule drug design and deep representation learning on drug development. The results of his research have been published in leading AI conferences, including AAAI, AISTATS, ICLR, IJCAI, KDD, NeurIPS, UAI, and top domain journals such as Nature, Cell Patterns, Nature Chemical Biology, and Bioinformatics. His work on clinical trial outcome prediction has been selected as the cover paper on Cell Patterns. In addition, Tianfan is an active community builder. He co-organized the first three AI4Science workshop on leading AI conferences (https://ai4sciencecommunity.github.io/); he co-founded Therapeutic Data Commons (TDC) initiative (https://tdcommons.ai/), an ecosystem with AI-solvable tasks, AI-ready datasets, and benchmarks in therapeutic science. Additional information is available at https://futianfan.github.io/.

Program specifications provide clear and precise descriptions of behaviors of a software system, which serves as a blueprint for its design and implementation. They help ensure that the system is built correctly and the functions work as intended, making it easier to troubleshoot, modify, and verify the system if needed. NIST suggests that the lack of high-quality specifications is the most common cause of software project failure. Nowadays, successful projects have an equal or even higher number of specifications than code (counted by lines).

In this talk, I will present my research on synthesizing both informal and formal specifications for software systems. I will explain how we use a combination of program and natural language semantics to automatically generate informal specifications, even for native methods without implementation in Java which previous methods could not handle. By leveraging the generated specifications, we successfully detect many code bugs and code-comment inconsistencies. Additionally, I will describe how we derive formal specifications from natural language comments using a search-based technique. The generated formal specifications have been applied to facilitate program analysis for existing tools. They have been shown to greatly improve the capabilities of these tools, by detecting many new information leaking paths and reducing false alarms in testing. Overall, the talk will highlight the importance of program specifications in software engineering and demonstrate the potential of our techniques to improve the development and maintenance of software systems.

Bio:

Juan Zhai is an Assistant Teaching Professor in the Department of Computer Science at Rutgers University. Previously, she was a Postdoctoral Research Associate, working with Prof. Xiangyu Zhang in the Department of Computer Science at Purdue University. She also worked as a tenure-track Assistant Professor at Nanjing University, where she obtained her Ph.D. degree. Her research interests lie in software engineering, natural language processing, and security, focusing on specification synthesis and enforcement. She is the recipient of the Distinguished Paper Award of USENIX Security 2017 and the Outstanding Doctoral Student Award in NASAC 2016.

Many learning tasks in Artificial Intelligence require dealing with graph data, ranging from biology and chemistry to finance and education. As powerful learning tools for graph inputs, graph neural networks (GNNs) have demonstrated remarkable performance in various applications. Despite their success, unlocking the full potential of GNNs requires tackling the limitations of robustness and scalability. In this talk, I will present a fresh perspective on enhancing GNNs by optimizing the graph data, rather than designing new models. Specifically, first, I will present a model-agnostic framework which improves prediction performance by enhancing the quality of an imperfect input graph. Then I will show how to significantly reduce the size of a graph dataset while preserving sufficient information for GNN training.

Traditionally, multimodal information consumption has been entity-centric with a focus on concrete concepts (such as objects, object types, physical relations, e.g., a person in a car), but lacks ability to understand abstract semantics (such as events and semantic roles of objects, e.g., driver, passenger, mechanic). However, such event-centric semantics are the core knowledge communicated, regardless whether in the form of text, images, videos, or other data modalities.

At the core of my research in Multimodal Information Extraction (IE) is to bring such deep semantic understanding ability to the multimodal world. My work opens up a new research direction Event-Centric Multimodal Knowledge Acquisition to transform traditional entity-centric single-modal knowledge into event-centric multi-modal knowledge. Such a transformation poses two significant challenges: (1) understanding multimodal semantic structures that are abstract (such as events and semantic roles of objects): I will present my solution of zero-shot cross-modal transfer (CLIP-Event), which is the first to model event semantic structures for vision-language pretraining, and supports zero-shot multimodal event extraction for the first time; (2) understanding long-horizon temporal dynamics: I will introduce Event Graph Model, which empowers machines to capture complex timelines, intertwined relations and multiple alternative outcomes. I will also show its positive results on long-standing open problems, such as timeline generation, meeting summarization, and question answering. Such Event-Centric Multimodal Knowledge Acquisition starts the next generation of information access, which allows us to effectively access historical scenarios and reason about the future. I will lay out how I plan to grow a deep semantic understanding of language world and vision world, moving from concrete to abstract, from static to dynamic, and ultimately from perception to cognition.

Bio: Manling Li is a Ph.D. candidate at the Computer Science Department of University of Illinois Urbana-Champaign. Her work on multimodal knowledge extraction won the ACL'20 Best Demo Paper Award, and the work on scientific information extraction from COVID literature won NAACL'21 Best Demo Paper Award. She was a recipient of Microsoft Research PhD Fellowship in 2021. She was selected as a DARPA Riser in 2022, and a EE CS Rising Star in 2022. She was awarded C.L. Dave and Jane W.S. Liu Award, and has been selected as a Mavis Future Faculty Fellow. She led 19 students to develop the UIUC information extraction system and ranked 1st in DARPA AIDA evaluation in 2019 and 2020. She has more than 30 publications on multimodal knowledge extraction and reasoning, and gave tutorials about event-centric multimodal knowledge at ACL'21, AAAI'21, NAACL'22, AAAI'23, etc. Additional information is available at https://limanling.github

The fuel of machine learning models and algorithms is the data usually collected from users, enabling refined search results, personalized product recommendations, informative ratings, and timely traffic data. However, increasing reliance on user data raises serious challenges. A common concern with many of these data-intensive applications centers on privacy — as a user’s data is harnessed, more and more information about her behavior and preferences is uncovered and potentially utilized by platforms and advertisers. These privacy costs necessitate adjusting the design of data markets to include privacy-preserving mechanisms. 

This talk establishes a framework for collecting data of privacy-sensitive strategic users for estimating a parameter of interest (by pooling users' data) in exchange for privacy guarantees and possible compensation for each user.  We formulate this question as a Bayesian-optimal mechanism design problem, in which an individual can share her data in exchange for compensation but at the same time has a private heterogeneous privacy cost which we quantify using differential privacy. We consider two popular data market architectures: central and local. In both settings, we use Le Cam's method to establish minimax lower bounds for the estimation error and derive (near) optimal estimators for given heterogeneous privacy loss levels for users. Next, we pose the mechanism design problem as the optimal selection of an estimator and payments that elicit truthful reporting of users' privacy sensitivities. We further develop efficient algorithmic mechanisms to solve this problem in both privacy settings.

Finally, we consider the case that users are interested in learning different personalized parameters. In particular, we highlight the connections between this problem and the meta-learning framework, allowing us to train a model that can be adapted to each user's objective function.

Bio:  Alireza Fallah is a Ph.D. candidate at the department of Electrical Engineering and Computer Science (EECS) and the Laboratory for Information and Decision Systems (LIDS) at Massachusetts Institute of Technology (MIT). His research interests are machine learning theory, data market and privacy, game theory, optimization, and statistics. He has received a number of awards and fellowships, including the Ernst A. Guillemin Best MIT EECS M.Sc. Thesis Award, Apple Scholars in AI/ML Ph.D. fellowship, MathWorks Engineering Fellowship, and Siebel Scholarship. He has also worked as a research intern at the Apple ML privacy team. Before joining MIT, he earned a dual B.Sc. degree in Electrical Engineering and Mathematics from Sharif University of Technology, Tehran, Iran.

Neural network models have been pushing computers’ capacity limit on natural language understanding and generation while lacking interpretability. The black-box nature of deep neural networks hinders humans from understanding their predictions and trusting them in real-world applications. In this talk, I will introduce my effort in bridging the trustworthy gap between models and humans by developing interpretation techniques, which cover three main phases of a model life cycle—training, testing, and debugging. I will demonstrate the critical values of integrating interpretability into every state of model development: (1) making model prediction behavior transparent and interpretable during training; (2) explaining and understanding model decision-making on each test example; (3) diagnosing and debugging models (e.g., robustness) based on interpretations. I will discuss future directions on incorporating interpretation techniques with system development and human interaction for long-term trustworthy AI.


Bio: Hanjie Chen is a Ph.D. candidate in Computer Science at the University of Virginia. Her research interests lie in Trustworthy AI, Natural Language Processing (NLP), and Interpretable Machine Learning. She is a recipient of the Carlos and Esther Farrar Fellowship and the Best Poster Award at the ACM CAPWIC 2021. Her work has been published at top-tier NLP/AI conferences (e.g., ACL, AAAI, EMNLP, NAACL) and selected by the National Center for Women & Information Technology (NCWIT) Collegiate Award Finalist 2021. Besides, as the primary instructor, she co-designed and taught a cross-listed course, CS 4501/6501 Interpretable Machine Learning, at UVA. Her effort in teaching was recognized by the UVA CS Outstanding Graduate Teaching Award and University-wide Graduate Teaching Awards Nominee (top 5% of graduate instructors). 

Abstract: The head of the Institute of Education Sciences has asked how we can use platform<https://www.the74million.org/article/schneider-garg-medical-researchers-find-cures-by-conducting-many-studies-and-failing-fast-we-need-to-do-the-same-for-education/>s to increase education sciences. We at ASSISTments have been addressing this<https://www.the74million.org/article/heffernan-how-can-we-know-if-ed-tech-works-by-encouraging-companies-researchers-to-share-data-government-funding-can-help/> very question. How do platforms like EdX, Khan Academy, or Canvas improve science? There is a crisis in American science referred to as the Reproducibility Crisis where many experimental results cannot be reproduced. We are trying to address that crisis by helping “good science” be done. People who control platforms have a responsibility to try to make them useful tools for learning what works. In Silicon Valley, every company is doing AB Testing to refine their individual products. That, in and of itself, is a good thing and we should use these platforms to figure out how to make them more effective. One of the ways we should do that is by experimenting with different ways of helping students succeed. ASSISTments<http://www.assistments.org/>, a platform I have created with 450,000 middle-school math students, is used to help scientists run studies. I will explain how we have over 100 experiments running inside the ASSISTments platform and how the ASSISTmentsTestBed.org allows external researchers to propose studies. I will also explain how proper oversight is done by our Institutional Review Board. Further, I will explain how users of this platform agree ahead of time to OpenScience procedures such as open-data, open-materials and pre-registration. I’ll illustrate some examples with the twenty-four randomized controlled trials<https://www.etrialstestbed.org/> that I have published as well as the three studies that have more recently come out from the platform by others. Finally, I will point to how we are anonymizing our data and how over 34 different external researchers have used our datasets to publish scientific studies<https://sites.google.com/site/assistmentsstudies/useourdata>. I would like to thank the U.S. Department of Education and the National Science Foundation for their support of over $32 million from 40+ grants.

Bio: Neil Heffernan is William Smith Dean's Professor of Computer Science and Director of the Learning Sciences & Technology program at Worcester Polytechnic Institute. He co-founded ASSISTments, a web-based learning platform, which he developed not only to help teachers be more effective in the classroom,  but also so that he could use the platform to conduct studies to improve the quality of education. Professor Heffernan is passionate about educational data mining and enjoys supervising WPI students, helping them create ASSISTments content and features. Several student projects have resulted in peer-reviewed publications that compare different ways to optimize student learning. Professor Heffernan's goal is to give ASSISTments to millions across the U.S. and internationally as a free service.

The presentation considers computer vision, especially a point of view of applications. Digital image processing and analysis with machine learning methods enable efficient solutions for various areas of useful data-centric engineering applications. Challenges with domain adaptation, active learning,  open set classification, and metric learning of similarities are considered. Computer Vision and Pattern Recognition Laboratory at LUT focuses on industrial computer vision, biomedical engineering, social signal processing, and data analytics. Different applications are given as examples based on specific data: fundus images in diagnosis of diabetic retinopathy, planktons in the Baltic Sea, Saimaa ringed seals in the Lake Saimaa, and logs in the sawmill industry.

Bio: Heikki Kälviäinen has been a Professor of Computer Science and Engineering since 1999. He is the head of the Computer Vision and Pattern Recognition Laboratory (CVPRL) at the Department of Computational Engineering of Lappeenranta-Lahti University of Technology LUT, Finland. He received his D.Sc. (Tech.) degree in Computer Science and Engineering in 1994 from the Department of Information Technology of LUT. Prof. Kälviäinen's research interests include computer vision, machine vision, pattern recognition, machine learning, and digital image processing and analysis.

The human mind is a remarkable thing. On the one hand, even "mundane" aspects of cognition, such as our ability to fold clean laundry, far outstrip the capabilities of the most advanced robotic systems. On the other hand, we know from decades of research in psychology that the human mind is also astonishingly limited. Human minds face strict limits on attention, working memory, perception, and other basic cognitive faculties. In this talk, I will discuss a framework for resolving this apparent tension that spans the fields of cognitive science and artificial intelligence, known as computational rationality. According to this framework, the mind utilizes computations and algorithms that maximize the expected utility of behavior, while subject to constraints on the ability to store, manipulate, and process information. In my research, I have focused primarily on the use of rate-distortion theory (a sub-field of information theory) to characterize these limits on human information processing. The approach will be demonstrated through computational models of perception, memory, decision making, and reinforcement learning.

Bio: Chris Sims received a B.S. in computer science from Cornell University (2003), followed by a Ph.D. in Cognitive Science from Rensselaer Polytechnic Institute (2009). After completing his Ph.D., Dr. Sims held a postdoctoral research position at the University of Rochester, and a faculty position at Drexel University before joining the faculty at RPI in 2017. Dr. Sims's overarching research interest lies in developing computational models of human intelligence. Specific research areas include reinforcement learning, visual memory and perceptual expertise, sensory-motor control and motor learning, and learning and decision-making under uncertainty.

Crowdsourcing studies how to aggregate a large collection of votes, which makes large-scale labeled datasets and modern AI possible. In this talk, I will mainly present our recent JAIR paper which focuses on labeling cost reduction problem of crowdsourcing. First, we reformulate the crowdsourcing problem as a statistical estimation problem where votes are approximated by worker models. Then doubly robust estimation is used to address this problem. We prove that the variance of estimation can be substantially reduced, hence the labeling cost can be reduced, even if the worker model is a poor approximation. Moreover, with adaptive worker/item selection rules, labeling cost can be further reduced. I conclude by summarizing future directions of doubly robust crowdsourcing and introducing more applications of voting, for example, private learning.

Bio:
Chong Liu is a PhD candidate in Computer Science at University of California, Santa Barbara (UCSB), advised by Prof. Yu-Xiang Wang. His research interests include ensemble learning, active learning, and global optimization. He has served on program committees of several conferences, e.g., AISTATS, AAAI, ICML, and NeurIPS. He is also serving on the editorial board of JMLR.

Networks (or graphs) are a powerful tool to model complex systems such as social networks, transportation networks, and the Web. The accurate modeling of such systems enables us to improve infrastructure, reduce conflicts in social media, and make better decisions in high-stakes settings. However, as graphs are highly combinatorial structures, these optimization and learning tasks require the design of efficient algorithms. 

In this talk, I will describe three research directions in the context of network data. First, I will overview several combinatorial problems for graph optimization that I have addressed using classical approaches such as approximate and randomized algorithms. The second part will focus on a different and a more recent approach to solving combinatorial problems by leveraging the power of machine learning. More specifically, I will show how combining neural architectures on graphs with reinforcement learning solves popular data ming problems such as the influence maximization problem.  In the last one, I will demonstrate how to deploy these methods on problems in computational social science with applications in decision-making for patent review systems and the stock market. 

Bio: Sourav Medya is a research assistant professor in the Kellogg School of Management at Northwestern University. He is also affiliated with the Northwestern Institute of Complex Systems. He has received his Ph.D. in Computer Science at the University of California, Santa Barbara. His research has been published at several venues including VLDB, NeurIPS, WebConf (WWW), AAMAS, IJCAI, WSDM, SDM, ICDM, SIGKDD Explorations, and TKDE. He has also been a PC member for WSDM, WebConf, AAAI, SDM, AAMAS, ICLR, and IJCAI. 

Sourav's research is focused on the problems at the intersection of graphs and machine learning. More specifically he designs data science tools that optimize graph-based processes and improve the quality as well as scalability of traditional graph combinatorial and mining problems. He also deploys these tools to solve problems in the interdisciplinary area of computational social science especially to improve innovation.

Large amounts of text are written and published daily. As a result, applications such as reading through the documents to automatically extract useful and structured information from the text have become increasingly needed for people’s efficient absorption of information. They are essential for applications such as answering user questions, information retrieval, and knowledge base population.

In this talk, I will focus on the challenges of finding and organizing information about events and introduce my research on leveraging knowledge and reasoning for document-level information extraction. In the first part, I’ll introduce methods for better modeling the knowledge from context: (1) generative learning of output structures that better model the dependency between extracted events to enable more coherent extraction of information (i.e., event A happening in the earlier part of the document is usually correlated with event B in the later part). (2) How to utilize information retrieval to enable memory-based learning with even longer context.

In the second part, to better access relevant external knowledge encoded in large models for reducing the cost of human annotations, we propose a new question-answering formulation for the extraction problem. I will conclude by outlining a research agenda for building the next generation of efficient and intelligent machine reading systems with close to human-level reasoning capabilities.

Bio:

Xinya Du is a Postdoctoral Research Associate at the University of Illinois at Urbana-Champaign working with Prof. Heng Ji. He earned a Ph.D. degree in Computer Science from Cornell University, advised by Prof. Claire Cardie. Before Cornell, he received a bachelor's degree in Computer Science from Shanghai Jiao Tong University. His research is on natural language processing, especially methods that leverage knowledge & reasoning skills for document-level information extraction. His work has been published in leading NLP conferences such as ACL, EMNLP, NAACL and has been covered by major media like New Scientist. He has received awards including the CDAC Spotlight Rising Star award and SJTU National Scholarship.

Quantum can solve complex problems that classical computers will never be able to.

In recent years, significant efforts have been devoted to building quantum computers to solve

real-world problems. To ensure the correctness of quantum systems, we develop the verification

techniques and testing algorithms for quantum systems. In the first part of this talk, I will overview

my work of efficient reasoning about quantum programs by developing verification techniques and

tools that leverage the power of Birkhoff & von Neumann quantum logic. In the second part, I will

review my work on quantum state tomography, i.e., learning the classical description of quantum

hardware, which closes a 40 years long-standing gap between the upper and lower bounds
for quantum state engineering.

Bio: Nengkun Yu is an associate professor in the Centre for Quantum Software and Information, 

the University of Technology Sydney. He received his B.S. and PhD degrees from the Department

of Computer Science and Technology, Tsinghua University, Beijing, China, in July of 2008 and 2013.

He won the ACM SIGPLAN distinguished paper award at OOPSLA 2020 and the ACM SIGPLAN

distinguished paper award at PLDI 2021. His research interest focuses on quantum computing.

Today, cryptography has transcended beyond protecting data in transit. Many advanced cryptographic techniques are gaining traction due to their significance in emerging systems like blockchains, privacy-preserving computation, and cloud computing. However, designing cryptography for these systems is challenging as they require underlying cryptographic algorithms to provide strong security and privacy guarantees and admit very efficient implementations.

In the first part of this talk, I will present my work that explores fundamental connections between cryptography and blockchains. Blockchains rely on advanced cryptography like Zero-knowledge Proofs, and despite amazing advances in building efficient cryptographic tools, scalability is a major challenge plaguing blockchain-based applications like cryptocurrencies. I will discuss my work on improving prover’s time and memory overheads in Zero-knowledge Proofs, which is currently a primary bottleneck towards building more efficient zero-knowledge proofs. Then, I will discuss my work where I use blockchains to build new and useful cryptography.

Finally, I will discuss my work on designing cryptographic commitments, digital analogs of sealed envelopes, secure against man-in-the-middle attacks. While heuristic constructions are known, my work introduces new fundamental techniques to circumvent strong barriers established for achieving provably secure protocols with minimal interaction. Specifically, I build provably secure protocols that require minimal (to no) interaction between the sender and the receiver.

Bio:
Pratik Soni is a Postdoctoral Research Fellow in the School of Computer Science at Carnegie Mellon University. He received his Ph.D. from UC Santa Barbara in 2015. His research interests span a wide range of topics across cryptography, including zero-knowledge proofs, non-malleable cryptography, secure multi-party computation, and its connections with blockchain technology. His work at FOCS 2017 was invited to SIAM Journal of Computing's special issue, and he is currently serving as a Program Committee Member at ACM CCS 2022 and ASIACRYPT 2022.

Our visual world is naturally open, containing visual elements that are dynamic, vast, and unpredictable. However, existing computer vision models are often developed inside a closed-world paradigm, for example, recognizing objects or human actions from a fixed set of categories. If these models are exposed to the realistic complexity of the open world, they will be brittle and fail to generalize. For example, an autonomous driving vehicle may not be able to avoid an accident if it sees a turnover truck, because it has never seen this novelty in its training data. In order to enable visual understanding in open world, vision models need to be aware of the unknown novelties, interpret their decision processes to human users, and adapt themselves to the novelties. In this talk, I will describe our recent work on open-set recognition and interpretable visual precognition. Together, these approaches make strides towards assurance autonomous AI in open world.

Bio: Dr. Yu Kong is now an Assistant Professor directing the ACTION lab in the Golisano College of Computing and Information Sciences at Rochester Institute of Technology. He received B.Eng. degree in automation from Anhui University in 2006, and PhD degree in computer science from Beijing Institute of Technology, China, in 2012. He was a postdoctoral research associate in the Department of Computer Science and Engineering, State University of New York, Buffalo in 2012, and then in the Department of Electrical and Computer Engineering, Northeastern University, Boston, MA. Dr. Kong's research in Computer Vision and Machine Learning is supported by National Science Foundation, Army Research Office, and Office of Naval Research, etc. His work has been publishing on top-tier conferences and transactions in the AI community such as CVPR, ECCV, ICCV, T-PAMI, IJCV, etc. He is an Associate Editor for Springer Journal of Multimedia Systems, and also serves as reviewers and PC members for prestige journals and conferences, including T-PAMI, T-IP, T-NNLS, T-CSVT, CVPR, ICLR, AAAI, and IJCAI, etc. More information can be found on his webpage at https://people.rit.edu/yukics/.