Blog

The Latest and Greatest from Salesforce Research

Aligning Diffusion Models to Human Preferences

TLDR Learning from human preferences, specifically Reinforcement Learning from Human Feedback (RLHF) has been a key recent component in the development of large language models such as ChatGPT or Llama2. Up until recently, the impact of human feedback training on text-to-image models was much more limited. In this work, Diffusion-DPO,

08 Jan 2024 • Bram Wallace #reinforcement-learning

The Ever-Growing Power of Small Models

Recent AI media coverage has followed a familiar pattern: a massive new model is released, making the rounds with beta testers and eventually the public, but it’s barely a month or two before rumors start to swell about the even bigger one supposedly being trained to replace it. Yet

21 Dec 2023 • Silvio Savarese

Salesforce Research at NeurIPS 2023

Conference Overview Next week, the Thirty-seventh annual Conference on Neural Information Processing Systems (NeurIPS) will be held in New Orleans, Louisiana from Sunday, December 10th, through Saturday, December 16th. NeurIPS will include invited talks, demonstrations, oral and poster presentations of accepted papers. NeurIPS 2023 will be held again at the

07 Dec 2023 • Mia Ferrer

BannerGen: A Library for Multi-Modality Banner Generation

Background Graphic layout designs serve as the foundation of communication between media designers and their target audience. They play a pivotal role in organizing various visual elements, including rendered text, logos, product images, calls to action (such as buttons), and background textures/images. The arrangement of these elements is the

06 Dec 2023 • Chia-Chih Chen

From Copilot to CoOrchestration

Einstein Copilot has arrived! Find out more about the conversational AI for CRM here. Introduction I’ve written a lot in recent months about what I call Large Action Models, or LAMs—a more active, autonomous variation on LLMs that don’t merely generate content like text or images but

20 Oct 2023 • Silvio Savarese

CodeChain: Towards Modular Code Generation through Chain of Self-revisions and Representative Sub-modules

TL;DR: With CodeChain, a pretrained large language model (LLM) can solve challenging coding problems by integrating modularity in generation samples and self-improve by employing a chain of self-revisions on representative sub-modules. CodeChain can achieve state-of-the-art results with both OpenAI GPT models and open-source LLMs on challenging coding benchmarks like

20 Oct 2023 • Henry Hung Le

Using language models to design antibodies to combat autoimmune disorders

TL;DR: We adapted our protein language model ProGen to optimize antibodies that bind to a protein called “CD40L”, a critical target for autoimmune disorders. We tested our AI designed antibodies in the laboratory and found that they bound very tightly to CD40L, showcasing the potential of this approach for

13 Oct 2023 • Ben Krause

GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation

Other authors include: Can Qin, Stefano Ermon, Yun Fu GlueGen was accepted by ICCV. In the rapidly advancing field of text-to-image synthesis, the remarkable progress in generating lifelike images from textual prompts has been evident. However, a significant challenge remains: how can we seamlessly integrate powerful pre-trained text encoders into

29 Sep 2023 • Ning Yu

Open Source and the Future of Enterprise AI

Einstein Copilot has arrived! Find out more about the conversational AI for CRM here. Introduction Open source has become one of the hottest topics in AI, and the fanfare is well-deserved. The open source community is keeping a nimble pace with the state of the art, delivering ever-growing and ever-more-capable

25 Sep 2023 • Silvio Savarese

We're Hiring! Trusted AI Roles at Salesforce

Meet the Office of Ethical and Humane Use Salesforce's Office of Ethical and Humane Use provides navigational guidance for the tough questions that arise when human potential meets emergent technology. We work across the company to guide the design, development, and deployment of trusted products, with a strong

21 Aug 2023 • Christina Zhang #AI ethics

Prototyping XGen-Image-1

TLDR Generative AI methods for image generation have a wide variety of potential applications in marketing, sales, and e-commerce. With these applications in mind, the Salesforce Research team has developed several techniques based on image-generative diffusion models, including methods for image editing, improved classifier guidance, and improved controlled generation methods.

03 Aug 2023 • Bram Wallace

PyRCA: Making Root Cause Analysis Easy in AIOps

TL;DR: PyRCA is an open-source machine learning library specifically designed for conducting Root Cause Analysis (RCA) in IT operations. It offers a comprehensive framework that allows users to easily identify the complicated metric causal dependencies and automatically locate the root causes of incidents. The library provides a unified interface

11 Jul 2023 • Chenghao Liu #root cause analysis

CodeGen2.5: Small, but mighty

Equal contribution between Erik Nijkamp and Hiroaki Hayashi. Paper Code Tweet Abstract The family of Salesforce CodeGen models is growing with CodeGen2.5 – a small, but mighty model! While there has been a recent trend of large language models (LLM) of increasing size, we show that a small model can

06 Jul 2023 • Erik Nijkamp #CodeGen

Toward Actionable Generative AI

LAMs: From Large Language Models to Large Action Models There’s no question that we’re living in the era of generative AI, and its impact is only growing. More and more, AI is helping us write emails, create imagery, consume information, and even code. But as empowering as it

27 Jun 2023 • Silvio Savarese

A Leap Forward in 3D Understanding: The ULIP and ULIP-2

TL;DR: Imagine a world where machines comprehend 3D objects just as humans do. The ULIP (CVPR2023) and ULIP-2 projects, backed by Salesforce AI, are making this a reality by revolutionizing 3D understanding. ULIP uniquely pre-trains models with 3D point clouds, images, and texts, aligning them into a unified representation

23 May 2023 • Le Xue

CodeT5+: Open Code Large Language Models

TL;DR: CodeT5+ is a new family of open code large language models (LLMs) with improved model architectures and training techniques. CodeT5+ achieves the state-of-the-art performance among the open-source LLMs on many challenging code intelligence tasks, including zero-shot evaluation on the code generation benchmark HumanEval.   Background: Code LLMs Large language

20 May 2023 • Yue Wang #codet5+

LogAI: A Library for Log Analytics and Intelligence

TL;DR LogAI is an open-source library designed for log analytics and intelligence. It can process raw logs generated by computer systems and support log analytics tasks such as log clustering and summarization, as well as log intelligence tasks such as log anomaly detection and root-cause analysis. LogAI is compatible

06 Apr 2023 • Doyen Sahoo

In Loving Memory of Dragomir Radev

The Salesforce AI Team is mourning the loss of our beloved friend and mentor, Dragomir Radev. Our team was first introduced to Drago in November 2018 when he gave a talk at our Research Speaker Series. His passion for research beamed through his talk and our leadership team unanimously decided

04 Apr 2023 • Audrey Cook

BotSIM: An End-to-End Automatic Evaluation Framework for Task-Oriented Dialog Systems

TL;DR: We present BotSIM, a data-efficient end-to-end Bot SIMulation toolkit for evaluation, diagnosis, and improvement of commercial task-oriented dialogue (TOD) systems. BotSIM's “generation-simulation-remediation'' paradigm can accelerate the end-to-end bot evaluation and iteration process by: (1) reducing the effort needed to create test cases; (2) enabling

29 Nov 2022 • Guangsen Wang #bot simulation

Salesforce AI Research at NeurIPS 2022

Conference Overview Next week, the Thirty-sixth annual Conference on Neural Information Processing Systems (NeurIPS) will be held in New Orleans, Louisiana from Monday, November 28th, through Friday, December 9th. NeurIPS will include invited talks, demonstrations, oral and poster presentations of accepted papers. Along with the conference is a professional exposition

22 Nov 2022 • Mia Ferrer

WarpDrive v2 Release Supports Numba to Simplify Machine Learning Workloads and Make Building Simulations Easier on NVIDIA GPUs

TL;DR: Deep reinforcement learning (RL), a powerful learning framework to train AI agents, can be slow as it requires repeated interaction with a simulation of the environment. Our original WarpDrive accelerates multi-agent deep RL on NVIDIA GPUs, enabling 10-100x speedups compared to alternative CPU+GPU implementations of multi-agent simulations.

02 Nov 2022 • Tian Lan #WarpDrive

DeepTime: Using Deep Time-Index Meta-Learning to Improve Non-Stationary Time-Series Forecasting

TL;DR: The performance of existing time-series forecasting methods can degrade due to non-stationarity, where the statistical distribution of time-series data changes over time. Our new DeepTime method overcomes non-stationarity issues by leveraging a “forecasting as meta-learning” framework on deep time-index models. DeepTime achieves competitive accuracy on the long-sequence time-series

13 Oct 2022 • Gerald Woo #DeepTime

Summer 2022 Salesforce Research Roundup

As we say a fond farewell to summer (bummer!), let's look back and review some of the stellar work reported on by Salesforce AI researchers during the past few months. (For more details, we encourage you to click the link for each project to read the full blog

30 Sep 2022 • Donald Rose #Summer 2022

Meet LAVIS: A One-stop Library for Language-Vision AI Research and Applications

TL;DR: LAVIS (short for LAnguage-VISion) is an open-source deep learning library for language-vision research and applications, offering comprehensive support for a wide range of tasks, datasets, and state-of-the-art models. Featuring a unified interface and modular design, it’s easy to use off-the-shelf and to extend with new capabilities. With

20 Sep 2022 • Dongxu Li #LAVIS

ETSformer: Exponential Smoothing Transformers for Time-Series Forecasting

TL;DR: We developed a new time-series forecasting model called ETSformer that leverages the power of two frameworks. By combining the classical intuition of seasonal-trend decomposition and exponential smoothing with modern transformers – as well as introducing novel exponential smoothing and frequency attention mechanisms – ETSformer achieves state-of-the-art performance. Background Before diving

23 Aug 2022 • Gerald Woo #ETSformer

AI for Global Climate Cooperation: Salesforce Research and Mila Announce Climate Change Collaboration and Competition

TL;DR:  Salesforce Research and Mila announce AI for Global Climate Cooperation, a working group collaboration and competition to design negotiation protocols and climate agreements. We plan to coauthor a peer-reviewed scientific paper with top-performing teams; insights will be distilled into a policy brief shared with leading policymakers, informing future

05 Aug 2022 • Stephan Zheng #AI for Global Climate Cooperation

AI Coding with CodeRL: Toward Mastering Program Synthesis with Deep Reinforcement Learning

TL;DR: CodeRL is a new framework for program synthesis through holistic integration of pretrained language models and deep reinforcement learning. By utilizing unit test feedback as part of model training and inference, and integrating with an improved CodeT5 model, CodeRL achieves state-of-the-art results on competition-level programming tasks. The following

19 Jul 2022 • Henry Hung Le #reinforcement-learning