Research | OpenAI
Our research on generative modeling for images has led to representation models like CLIP, which makes a map between text and images that an AI can read, and DALL-E, a tool for creating vivid images from text descriptions.
Research - OpenAI
Explore the research we're conducting to stay at the forefront of AI development and deployment.
In this paper, we explore a semi-supervised approach for language understanding tasks using a combination of unsupervised pre-training and supervised fine-tuning. Our goal is to learn a universal representation that transfers with little adaptation to a …
Weak-to-strong generalization - OpenAI
Dec 14, 2023 · Today, we are releasing the team’s first paper, which introduces a new research direction for empirically aligning superhuman models. Current alignment methods, such as reinforcement learning from human feedback (RLHF), rely on human supervision.
Video generation models as world simulators | OpenAI
Feb 15, 2024 · We explore large-scale training of generative models on video data. Specifically, we train text-conditional diffusion models jointly on videos and images of variable durations, resolutions and aspect ratios. We leverage a transformer architecture that operates on spacetime patches of video and image latent codes.
CLIP: Connecting text and images | OpenAI
Jan 5, 2021 · 1. CLIP is highly efficient. CLIP learns from unfiltered, highly varied, and highly noisy data, and is intended to be used in a zero-shot manner. We know from GPT-2 and 3 that models trained on such data can achieve compelling zero shot performance; however, such models require significant training compute.
GPTs are GPTs: An early look at the labor market impact ... - OpenAI
Mar 17, 2023 · We investigate the potential implications of Generative Pre-trained Transformer (GPT) models and related technologies on the U.S. labor market. Using a new rubric, we assess occupations based on their correspondence with GPT capabilities, incorporating both human expertise and classifications from GPT-4.
In contrast to a lot of work on speech recognition, we train Whisper models to predict the raw text of transcripts without any significant standardization, relying on the expressive-ness of sequence-to-sequence models to learn to map be-tween utterances and …
Improving language understanding with unsupervised learning
Jun 11, 2018 · Improving language understanding with unsupervised learning. Read paper. Illustration: Ben Barry. We’ve obtained state-of-the-art results on a suite of diverse language tasks with a scalable, task-agnostic system, which we’re also releasing. Our approach is a combination of two existing ideas: transformers and unsupervised pre-training.
Aligning language models to follow instructions | OpenAI
Jan 27, 2022 · The OpenAI API is powered by GPT-3 language models which can be coaxed to perform natural language tasks using carefully engineered text prompts. But these models can also generate outputs that are untruthful, toxic, or reflect harmful sentiments.