TL; DR: Introducing CodeT5 --- the first code-aware, encoder-decoder-based pre-trained programming language model, which enables a wide range of code intelligence applications including code understanding and generation tasks. CodeT5 achieves state-of-the-art performance on 14 sub-tasks in the CodeXGLUE code intelligence benchmark. Given the goal of improving software development productivity with

tldr: WarpDrive is an open-source framework to do multi-agent RL end-to-end on a GPU. It achieves orders of magnitude faster multi-agent RL training with 2000 environments and 1000 agents in a simple Tag environment. WarpDrive provides lightweight tools and workflow objects to build your own fast RL workflows. Check out

This blog accompanies the interactive demo [https://einstein.ai/the-ai-economist/ai-policy-foundation-and-covid-case-study] and paper [https://arxiv.org/abs/2108.02904]! Dive deeper into the underlying simulation code [https://www.github.com/salesforce/ai-economist ] and simulation card [https://github.com/salesforce/ai-economist/blob/master/COVID-19_Simulation-Card.pdf] . In this blog post, we

August 5, 2021 In June 2021, four of us from the Salesforce AI Ethics and Conversational Design teams collaborated with the Montreal AI Ethics Institute [https://montrealethics.ai/] (MAIEI) to facilitate a workshop on the responsible creation and implementation of chatbots and conversational assistants. Connor Wright [https://www.linkedin.com/

AI can learn the language of biology to create artificial proteins across multiple protein types that are functional and unseen in nature.

> TL; DR: We propose a new vision-language representation learning framework which achieves state-of-the-art performance by first aligning the unimodal representations before fusing them. Vision and language are two of the most fundamental channels for humans to perceive the world. It has been a long-standing goal in AI to build intelligent

An AI model can generate sequences across natural language and proteins with attributes that go beyond the training distribution

With data rapidly being generated by millions of people, it's not feasible to label all of it. Learn about the recent advancements in ML for how to train vision models with unlabelled data using self-supervised learning.

How many emails and working-related conversations do you have every day? The average office worker receives about 121 emails [https://www.campaignmonitor.com/blog/email-marketing/2019/05/shocking-truth-about-how-many-emails-sent/] daily and uncountable messages on platforms such as Slack [https://slack.com/], Team [https://www.microsoft.com/en-us/microsoft-teams/group-chat-software], or iMessage

This year marks the 9th annual conference on International Conference on Learning Representations (ICLR) taking place in a fully virtual format from May 4th through May 8th, 2021. ICLR is a premier academic conference in the field of representation learning, generally referred to as deep learning or feature learning. ICLR

The empirical success of deep learning has posed significant challenges to machine learning theory: Why can we efficiently train neural networks with gradient descent despite its highly non-convex optimization landscape? Why do over-parametrized networks generalize well? The recently proposed Neural Tangent Kernel (NTK) theory offers a powerful framework for understanding

> TL;DR: We propose controllable counterfactuals (CoCo) to evaluate dialogue state tracking (DST) models on novel scenarios, which results in significant performance drop of up to 30.8% for state-of-the-art DST models. Using CoCo for data augmentation improves the accuracy of the SoTA DST model by 5.5% on the

Transformer-inspired tips for enhancing the design of neural networks that process graph-structured data

February 2020 Summary from FAccT 2020 CRAFT Session > AI Ethics practitioners in industry look to researchers for insights on how to best identify and mitigate harmful bias in their organization’s AI solutions and create more fair or equitable outcomes. However, it can be a challenge to apply those research

We are proud to announce the 2020 winners of our Salesforce AI Research Grant. Each of our winners will receive a $50K grant to advance their work and help us shape the future of AI.

AI makes breast cancer therapy decisions more accurate, affordable and accessible using visual clues in pathology images invisible to the eye

> TL; DR: We find that current self-supervised learning approaches suffer from poor visual grounding and receive improper supervisory signal when trained on complex scene images. We introduce CAST to improve visual grounding during pretraining and show that it yields significantly better transferable features. Self-supervised learning and its grounding problem Self-Supervised

This year marks the 34th annual conference on Neural Information Processing Systems (NeurIPS [https://neurips.cc/]) reimagined for the first time ever in a fully virtual format. NeurIPS is a leading conference in the area of machine learning and neural information processing systems in their biological, technological, mathematical, and theoretical

> TL; DR: We propose a new semi-supervised learning method which achieves state-of-the-art performance by learning jointly-evolved class probabilities and image representations. What are the existing semi-supervised learning methods? Semi-supervised learning aims to leverage few labeled data and a large amount of unlabeled data. As a long-standing and widely-studied topic in

This year marks the 24th annual Empirical Methods in Natural Language Processing (EMNLP) [https://2020.emnlp.org/] conference reimagined for the first time ever in a fully virtual format. EMNLP is a leading conference in the area of Natural Language Processing covering a broad spectrum of diverse research areas that

In recent years, the natural language processing (NLP) community has seen the development of increasingly powerful language models [1, 2], capable of generating textual output that is indistinguishable from human-written text. This includes our own model called CTRL [https://blog.salesforceairesearch.com/introducing-a-conditional-transformer-language-model-for-controllable-generation/] [3] (Conditional Transformer Language Model) for controllable

We recently released Foundation, an open-source framework [https://github.com/salesforce/ai-economist] to build economic simulations. Foundation has been designed with flexibility and AI research in mind, and can be modified by anyone. AI simulations offer researchers the power to generate data and evaluate outcomes of virtual economies that capture

At Salesforce, we take seriously our mission to create and deliver AI technology that is responsible, accountable, transparent, empowering, and inclusive. These principles ensure that our AI is safe, ethical, and engenders trust.

> TL;DR: We theoretically analyze the differential architecture search (DARTS) for understanding the role and impact of skip connections, which inspires a new method for Neural Architecture Search (NAS) using group-structured sparse gates and path-depth-wise regularization to overcome the limitation of existing NAS methods for AutoML. In our work [1]

We use smaller language models as generative classifiers to guide generation from larger language models. We show that this method can make generations friendlier, reduce bias and toxicity, and achieve zero-shot controllable generation of unseen topics.

> TL; DR: We propose a new webly-supervised learning method which achieves state-of-the-art representation learning performance by training on large amounts of freely available noisy web images. Deep neural networks are known to be hungry for labeled data. Current state-of-the-art CNNs are trained with supervised learning on datasets such as ImageNet

We introduce Photon, a live demo of natural language interface to databases based on our latest research in neural semantic parsing. 🔗 https://naturalsql.com/

For all the good that AI can bring, responsible tech companies understand they must recognize, prepare for, and mitigate the potential unintended, harmful effects. That’s why Salesforce sees ethics as foundational to AI — and why we’re sharing a closer look at how we infuse an ethical process into

August 3 - 7, 2020. Fully virtual sessions. At Salesforce Research Asia, we aim to play a more active role in raising awareness to the exciting possibilities surrounding the use of AI in real-world application domains, data innovation challenges and issues, as well as to highlight our work in the

Our Salesforce Research team is inviting submissions from university faculty, non-profit organizations, and NGOs to apply for our Salesforce AI Research Grant.

We are launching an open source collaborative project to build an AI Economist that can be used to guide policy making in the real world. We invite you to join us in our mission to help improve the world with AI and economics.

We propose a simple causal (unidirectional) language model for Task-oriented Dialogue. SimpleTOD enables modeling of the inherent dependencies between the sub-tasks of task-oriented dialogue, by optimizing for all tasks in an end-to-end manner.

Morpheus exposes the potential allocative harms of popular pretrained NLP models by simulating inflectional variation. We propose adversarial fine-tuning for mitigating the effects of training only on error-free Standard English data.

Sneak a peek into our Industrial PhD Program with Samson.

The 58th Association for Computational Linguistics [https://acl2020.org/] (ACL) Conference kicked off this week and runs from Sunday, Jul 5 to Friday, Jul 10 in a fully virtual format. ACL is the premier conference of the field of computational linguistics, covering a broad spectrum of diverse research areas that

Word embeddings inherit strong gender bias in data which can be further amplified by downstream models. We propose to purify word embeddings against corpus regularities such as word frequency prior to inferring and removing the gender subspace, which significantly improves the debiasing performance.

In our study, we show how a language model, trained simply to predict a masked (hidden) amino acid in a protein sequence, recovers high-level structural and functional properties of proteins.

Prototypical Contrastive Learning unifies clustering and contrastive self-supervised learning to push the frontiers of unsupervised learning.

We show that deep neural models can describe common sense physics in a valid and sufficient way that is also generalizable. Our ESPRIT framework is trained on a new dataset with physics simulations and descriptions that we collected and have open-sourced.

In this work, we focus on the opportunity to use AI to promote social welfare through the design of optimal tax policies in dynamic economies.

Summary We investigate NVIDIA's Triton (TensorRT) Inference Server [https://docs.nvidia.com/deeplearning/sdk/triton-inference-server-guide/docs/index.html] as a way of hosting Transformer Language Models. The blog is roughly divided into two parts: (i) instructions for setting up your own inference server, and (ii) benchmarking experiments. The instructions are

In our study, we demonstrate that an artificial intelligence (AI) model can learn the language of biology in order to generate proteins in a controllable fashion.

We are proud and excited to announce this year’s winners of the Salesforce Research Deep Learning Grant.

Our graph-based trainable retriever-reader framework retrieves evidence paragraphs from Wikipedia to answer open-domain questions. We show state-of-the-art performance on HotpotQA, SQuAD Open, and Natural Questions Open without any architectural changes.

Learn about the ethical implications of voice for business and how to make them an operational and strategic priority now—before you’re too far down the path.

Many NLP applications today deploy state-of-the-art deep neural networks that are essentially black-boxes. One of the goals of Explainable AI (XAI) is to have AI models reveal why and how they make their predictions so that these predictions are interpretable by a human. But work in this direction has been

Large-scale language models show promising text generation capabilities, but users cannot control their generated content, style or train them for multiple supervised language generation tasks.

The Salesforce Research team announces their second annual Deep Learning Grant to fund AI research projects from around the world.

Earlier this year, Danielle Cass and I ran a workshop with 23 ethical AI practitioners from 15 organizations and shared out insights of what they are doing that has been successful and the open questions or challenges they are working through. A few months later, Matt Marshall, Founder of VentureBeat,

Published: July 10, 2019 As a discipline, those of us working on ethical or responsible AI, are learning together how to translate ethical principles into business practices that work for each of our organizations. The result is a shared community of practice. In this blog post, we share early lessons