This feature will allow users to have a voice conversation with ChatGPT or show what they are talking about. Until now, ChatGPT is only limited to text form where you can give information only in text input. It was simply a text based based chatbot. But with this new feature, it is now multimodal.
What ChatGPT can do with the new Feature
With the rollout of the new feature, chatGPT gets more powerful, and interactive. It can now speak, hear, see, and more! 👇
- Voice Capabilities: ChatGPT now allows users to have voice conversations, providing a more dynamic and engaging interface.
- Image Interaction: Users can show ChatGPT images, like landmarks or items in their home, to get insights or suggestions. This includes using the drawing tool on the mobile app!
- New Text-to-Speech Model: The voice functionality is powered by a new model that can produce human-like audio from just text and a short sample of speech.
- Collaboration with Spotify: The new voice technology is being used in Spotify's Voice Translation feature pilot, allowing podcasters to translate their content into additional languages in their own voices.
ChatGPT can now see, hear, and speak. Rolling out over next two weeks, Plus users will be able to have voice conversations with ChatGPT (iOS & Android) and to include images in conversations (all platforms). https://t.co/uNZjgbR5Bm pic.twitter.com/paG0hMshXb
— OpenAI (@OpenAI) September 25, 2023
“Snap a picture of a landmark while traveling and have a live conversation about what’s interesting about it,” the OpenAI blog said.
OpenAI is rolling out this new update in the coming two weeks for ChatGPT Plus and Enterprise users. In the meantime, the voice feature will only be available on iOS and Android while images will be available on all platforms.
How to Start Voice Conversation with ChatGPT
As we have mentioned above, the voice feature will only be available on iOS and Android while images will be available on all platforms.
Step 1: To get started with voice, head to Settings → New Features on the mobile app and opt into voice conversations.
Step 2: Then, tap the headphone button located in the top-right corner of the home screen and choose your preferred voice out of five different voices.
Step 3: The new voice capability is powered by a new text-to-speech model, capable of generating human-like audio from just text and a few seconds of sample speech.
How to Start Chat with Image on ChatGPT
Step 1: To get started, tap the photo button to capture or choose an image. If you’re on iOS or Android, tap the plus button first.
Step 2: You can also discuss multiple images or use our drawing tool to guide your assistant.
Step 3: Image understanding is powered by multimodal GPT-3.5 and GPT-4. These models apply their language reasoning skills to a wide range of images, such as photographs, screenshots, and documents containing both text and images.