ChatGPT (Chat Generative Pre-trained Transformer) is a chatbot developed by OpenAI and released in November 2022. It builds  and refines  OpenAI’s GPT-3 family of large-scale language models (an approach to transfer learning). ) using supervised reinforcement learning techniques.ChatGPT – Generative Pretrained Transformer (GPT) – enhanced with GPT-3.5 using supervised learning and reinforcement learning (an approach to transfer learning). Both approaches used human trainers to improve model performance.

In supervised learning, the model was provided with a conversation in which the trainer acted as both a user and an AI assistant. In the reinforcement learning step, the human trainer first sorted the answers given by the model  in previous conversations. These rankings were used to create a “reward model” in which multiple iterations of Proximal Policy Optimization (PPO) were performed to further refine the model.

Approximate policy optimization algorithms have the advantage of being more cost-effective than confidence interval policy optimization algorithms. They disable many computationally intensive operations with faster performance. This model was trained by Microsoft on Azure supercomputing infrastructure.Additionally, OpenAI will continue to collect data from ChatGPT users that can be used to further train and tune ChatGPT. Users can vote for or against responses received from ChatGPT. You can also provide additional feedback in the text box when upvoting or voting.