Equal contribution between Erik Nijkamp and Hiroaki Hayashi. Paper Code Tweet Abstract The family of Salesforce CodeGen models is growing with CodeGen2.5 – a small, but mighty model! While there has been a recent trend of large language models (LLM) of increasing size, we show that a small model can

TLDR We trained a series of 7B LLMs named XGen-7B with standard dense attention on up to 8K sequence length for up to 1.5T tokens. We also fine tune the models on public-domain instructional data. The main take-aways are: * On standard NLP benchmarks, XGen achieves comparable or better results

Links: Research Paper [https://arxiv.org/abs/2203.13474], Github [https://github.com/salesforce/CodeGen] -------------------------------------------------------------------------------- Can you imagine a machine writing an app for you, just by telling it what you want? As futuristic as this scenario sounds, it’s actually here today. Salesforce AI Research outlines conversational AI