The Latest and Greatest from Salesforce Research

BotSIM: An End-to-End Automatic Evaluation Framework for Task-Oriented Dialog Systems

TL;DR: We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for evaluation, diagnosis, and improvement of commercial task-oriented dialogue (TOD) systems. BotSIM's “generation-simulation-remediation'' paradigm can accelerate the end-to-end bot evaluation and iteration process by: (1) reducing the effort needed to create test cases; (2) enabling a better understanding of

29 Nov 2022 • Guangsen Wang #bot simulation

Salesforce AI Research at NeurIPS 2022

Conference Overview Next week, the Thirty-sixth annual Conference on Neural Information Processing Systems (NeurIPS) will be held in New Orleans, Louisiana from Monday, November 28th, through Friday, December 9th. NeurIPS will include invited talks, demonstrations, oral and poster presentations of accepted papers. Along with the conference is a professional exposition

22 Nov 2022 • Mia Ferrer

WarpDrive v2 Release Supports Numba to Simplify Machine Learning Workloads and Make Building Simulations Easier on NVIDIA GPUs

TL;DR: Deep reinforcement learning (RL), a powerful learning framework to train AI agents, can be slow as it requires repeated interaction with a simulation of the environment. Our original WarpDrive [] accelerates multi-agent deep RL on NVIDIA GPUs, enabling 10-100x speedups compared to alternative CPU+

02 Nov 2022 • Tian Lan #WarpDrive

DeepTime: Using Deep Time-Index Meta-Learning to Improve Non-Stationary Time-Series Forecasting

TL;DR: The performance of existing time-series forecasting methods can degrade due to non-stationarity, where the statistical distribution of time-series data changes over time. Our new DeepTime method overcomes non-stationarity issues by leveraging a “forecasting as meta-learning” framework on deep time-index models. DeepTime achieves competitive accuracy on the long-sequence time-series

13 Oct 2022 • Gerald Woo #DeepTime

Summer 2022 Salesforce Research Roundup

As we say a fond farewell to summer (bummer!), let's look back and review some of the stellar work reported on by Salesforce AI researchers during the past few months. (For more details, we encourage you to click the link for each project to read the full blog post.) --------------------------------------------------------------------------------

30 Sep 2022 • Donald Rose #Summer 2022

Meet LAVIS: A One-stop Library for Language-Vision AI Research and Applications

TL;DR: LAVIS (short for LAnguage-VISion) is an open-source deep learning library for language-vision research and applications, offering comprehensive support for a wide range of tasks, datasets, and state-of-the-art models. Featuring a unified interface and modular design, it’s easy to use off-the-shelf and to extend with new capabilities. With

20 Sep 2022 • Dongxu Li #LAVIS

ETSformer: Exponential Smoothing Transformers for Time-Series Forecasting

TL;DR: We developed a new time-series forecasting model called ETSformer that leverages the power of two frameworks. By combining the classical intuition of seasonal-trend decomposition and exponential smoothing with modern transformers – as well as introducing novel exponential smoothing and frequency attention mechanisms – ETSformer achieves state-of-the-art performance. Background Before diving

23 Aug 2022 • Gerald Woo #ETSformer

AI for Global Climate Cooperation: Salesforce Research and Mila Announce Climate Change Collaboration and Competition

TL;DR:  Salesforce Research and Mila announce AI for Global Climate Cooperation, a working group collaboration and competition to design negotiation protocols and climate agreements. We plan to coauthor a peer-reviewed scientific paper with top-performing teams; insights will be distilled into a policy brief shared with leading policymakers, informing future

05 Aug 2022 • Stephan Zheng #AI for Global Climate Cooperation

AI Coding with CodeRL: Toward Mastering Program Synthesis with Deep Reinforcement Learning

TL;DR: CodeRL is a new framework for program synthesis through holistic integration of pretrained language models and deep reinforcement learning. By utilizing unit test feedback as part of model training and inference, and integrating with an improved CodeT5 model, CodeRL achieves state-of-the-art results on competition-level programming tasks. The following

19 Jul 2022 • Henry Hung Le #reinforcement-learning

Salesforce Research at ICML 2022

Conference Overview This weekend will kick off the thirty-ninth International Conference on Machine Learning (ICML). This conference specifically aims to bring together professionals who are dedicated to the advancement of Machine Learning (ML) in Artificial Intelligence. Participants at ICML come from many different backgrounds, including academic and industrial researchers, entrepreneurs

17 Jul 2022 • Mia Ferrer #conferences

Salesforce Research at NAACL 2022

Conference Overview This weekend marks the start of the Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL). NAACL provides a regional focus for members of the Association for Computational Linguistics (ACL) in North America. NAACL organizes annual conferences, promotes cooperation and information exchange among

10 Jul 2022 • Mia Ferrer #NAACL 2022

Salesforce Research at CVPR 2022

Conference Overview The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) is the annual conference on Computer Vision. CVPR is composed of both the main conference, as well as workshops and other courses, to provide a unique learning experience and networking opportunities in the field of Computer Vision. CVPR

20 Jun 2022 • Mia Ferrer #computer vision

TaiChi: Open Source Library for Few-Shot NLP

AUTHORS: Sharvin Shah, Jin Qu, Donald Rose TL;DR: TaiChi is an open source library for few-shot NLP, designed for data scientists and software engineers who want to get some quick results or build proof-of-concept products but don’t have much experience with few-shot learning (FSL). The library abstracts complex

15 Jun 2022 • Jin Qu #NLP

Turbocharge Multi-Agent Reinforcement Learning with WarpDrive and PyTorch Lightning

TL;DR: WarpDrive is a flexible, lightweight, easy-to-use end-to-end reinforcement learning (RL) framework; enables orders-of-magnitude faster training on a single GPU. PyTorch Lightning enables you to modularize experimental code, and build production-ready workloads fast. Together, they can help significantly accelerate multi-agent RL R&D. Reinforcement Learning: Agents Learn by Maximizing

20 May 2022 • Sunil Srinivasa #WarpDrive

Salesforce Research at ACL 2022

Conference Overview This year marks the 60th annual meeting of the Association for Computational Linguistics Conference (ACL []). ACL is the premier international scientific and professional society for people working on computational problems involving human language, a field often referred to as either computational linguistics or

19 May 2022 • Mia Ferrer #NLP

Science Advances Publishes AI Economist Research on Improving Tax Policies With Reinforcement Learning

TL;DR: The AI Economist, a reinforcement learning (RL) system, learns dynamic tax policies that optimize equality along with productivity in simulated economies, outperforming alternative tax systems. We have now expanded this research, which is being published in the interdisciplinary scientific journal Science Advances. Humans or AI: Which Can Design

05 May 2022 • Stephan Zheng #AI Economist

Salesforce Research at ICLR 2022

Conference Overview This year marks the Tenth International Conference on Learning Representations ( ICLR []), one of the premier academic conferences dedicated to advancing research in representation learning - a type of machine learning also referred to as feature learning or deep learning. ICLR features the latest

25 Apr 2022 • Mia Ferrer #ICLR

Conversational AI Programming with CodeGen: Let AI Write Code For You

Links: Research Paper [], Github [] -------------------------------------------------------------------------------- Can you imagine a machine writing an app for you, just by telling it what you want? As futuristic as this scenario sounds, it’s actually here today. Salesforce AI Research outlines conversational AI

29 Mar 2022 • Erik Nijkamp #conversational AI

Embracing Ethical AI at NeurIPS 2021

December 21, 2021 The leading AI research conference, NeurIPS 2021, has recently wrapped up, spanning seven very full days, 2,344 accepted papers, eight invited talks, ten tutorials, and nearly 70 workshops. Though there was diverse and innovative thought leadership on display, I found myself drawn to the particular topics

21 Dec 2021 • Anna Bethke #AI-fairness

CodeT5: The Code-aware Encoder-Decoder based Pre-trained Programming Language Models

TL; DR: Introducing CodeT5 --- the first code-aware, encoder-decoder-based pre-trained programming language model, which enables a wide range of code intelligence applications including code understanding and generation tasks. CodeT5 achieves state-of-the-art performance on 14 sub-tasks in the CodeXGLUE code intelligence benchmark. -------------------------------------------------------------------------------- CodeT5 for code-related understanding and generation tasksGiven the

03 Sep 2021 • Yue Wang #code-intelligence

Data-Driven, Interpretable, and Robust Policy Design using the AI Economist

This blog accompanies the interactive demo [] and paper []! Dive deeper into the underlying simulation code [ ] and simulation card [] . In this blog post, we

09 Aug 2021 • Stephan Zheng

"The Triangle of Trust in Conversational Ethics and Design: Where Bots, Language and AI Intersect" Workshop Summary

August 5, 2021 In June 2021, four of us from the Salesforce AI Ethics and Conversational Design teams collaborated with the Montreal AI Ethics Institute [] (MAIEI) to facilitate a workshop on the responsible creation and implementation of chatbots and conversational assistants. Connor Wright [

05 Aug 2021 • Yoav Schlesinger #ethics

Learning without Labels

With data rapidly being generated by millions of people, it's not feasible to label all of it. Learn about the recent advancements in ML for how to train vision models with unlabelled data using self-supervised learning.

21 Jun 2021 • Michael Sollami #deeplearning

Slack your way to QA - How past conversations can answer future questions.

How many emails and working-related conversations do you have every day? The average office worker receives about 121 emails [] daily and uncountable messages on platforms such as Slack [], Team [], or iMessage

07 Jun 2021 • Chien-Sheng Wu

Salesforce Research at ICLR 2021

This year marks the 9th annual conference on International Conference on Learning Representations (ICLR) taking place in a fully virtual format from May 4th through May 8th, 2021. ICLR is a premier academic conference in the field of representation learning, generally referred to as deep learning or feature learning. ICLR

26 Apr 2021 • Mia Ferrer #ICLR

When are Neural Networks more powerful than Neural Tangent Kernels?

The empirical success of deep learning has posed significant challenges to machine learning theory: Why can we efficiently train neural networks with gradient descent despite its highly non-convex optimization landscape? Why do over-parametrized networks generalize well? The recently proposed Neural Tangent Kernel (NTK) theory offers a powerful framework for understanding

29 Mar 2021 • Yu Bai #deep learning theory