Recently, a test was conducted to see which one, between ChatGPT and Gemini, Google's AI, deserves to take the top spot.
After ChatGPT was released and quickly gained public adoption in November 2022, other tech giants immediately jumped into this AI race, with Google being no exception.
This guide explains the key differences between ChatGPT and Gemini chat based on tests conducted in March 2024. This head-to-head comparison between these two heavyweights will provide us with the following insights:
- Gemini vs. ChatGPT: What are the differences?
- Gemini vs. ChatGPT test: Key findings
- Gemini vs. ChatGPT - free version: Test results
- Gemini Advanced vs. ChatGPT Plus - paid version: Test results
- Gemini Advanced vs. ChatGPT Plus: AI images
- Gemini vs. ChatGPT: User experience
1. Gemini vs. ChatGPT: What are the differences?
For beginners who don't know what Google Bard is or What is Google Gemini. Gemini, is an upgrade and rebrand of Bard. In March 2023, Google Bard AI was launched in the United States and the United Kingdom. Google Bard is an intelligent AI chatbot that can understand and quickly answer your questions based on a large database. Later, Google Bard was renamed to Gemini and a paid version was released.
1.1. Differences in Large Language Models (LLM)
We already know what Google Bard is . Returning to the main difference between Gemini and ChatGPT, it lies in the Large Language Model (LLM) they use and their respective data sources. Gemini Google AI has been powered by several different language models since its launch, while ChatGPT users have utilized GPT-3, GPT-3.5, and GPT-4.
Google Bard was initially powered by LaMDA and more recently by PaLM 2. PaLM 2 can reason in over 100 languages and its training set includes significantly more code than LaMDA. Thanks to PaLM 2, Google Bard AI has become better at writing code in programming languages like Python. Other information used to train PaLM 2 includes scientific papers, mathematical expressions, and source code.
In December 2023, Google released Gemini Pro. Testing showed that Gemini Pro outperformed PaLM 2, and early reports indicated it was more helpful in providing answers to coding queries as well as text-based tasks. Since then, the company has released Gemini Ultra to power the new Gemini Advanced chatbot.
The Gemini language models from Google AI – Pro, Ultra, and Nano – have been trained on a wide variety of inputs, not just text. Google has also fine-tuned the models with more multimodal information. They can access the Internet in real-time to process tasks efficiently and accurately.
Meanwhile, ChatGPT is powered by the GPT language model developed by OpenAI. GPT-3.5 powers the free version of ChatGPT . It was trained on a predefined dataset and has not been updated since January 2022 (originally September 2021). ChatGPT was trained on data such as Common Crawl, Wikipedia, articles, and documents, similar to Gemini. The next version of ChatGPT is GPT-4, which is exclusive to ChatGPT Plus subscribers and was trained on a larger dataset.
1.2. Gemini vs. ChatGPT: Other key differences
Beyond the differences in data sources and language models, there are several other key distinctions between Gemini and ChatGPT as follows:
- Learning through conversation: ChatGPT can learn from user conversations and "grasp context," whereas Gemini does this in a much more limited way at the moment.
- Drafts: Gemini generates multiple responses to your queries. You can view other responses by clicking "View Drafts." Meanwhile, ChatGPT only generates a single response for each query.
- Editing responses: Gemini chat allows you to edit responses after you send them, whereas you cannot do this with ChatGPT.
- Real-time Internet access: Gemini AI chat always has access to real-time information from the Internet, while ChatGPT has only recently gained this functionality and it is not available on the free version.
- Image-based responses: Gemini always has Internet access, so it can retrieve images from Google. Bard Advanced and ChatGPT Plus users can currently generate images using AI.
- Text-to-speech capability: Gemini has the ability to convert text into natural-sounding speech, while ChatGPT not really.
2. Google Gemini vs. ChatGPT: Key Findings
Before diving into how the free and paid versions of Gemini and ChatGPT answer questions, here are some of the most notable takeaways from using these two AI chatbots:
- Responses from Gemini are more conversational, while responses from ChatGPT are more informative.
- The free version of Gemini is better than the free version of ChatGPT.
- However, Gemini Advanced is only slightly better than ChatGPT Plus.
- Gemini offers a better user experience with more images and website links.
- Gemini Advanced generates better AI images than ChatGPT Plus.
- Gemini's responses are generally presented in a more readable format than those of ChatGPT.
- Gemini create better spreadsheet formulas ChatGPT
3. Gemini vs. ChatGPT - free versions: Test results
I asked 12 questions to the free version of Gemini Google's AI and ChatGPT from OpenAI. The prompts were selected to showcase each chatbot's ability to handle a wide variety of queries:
- Self-awareness
- Ethical reasoning
- Conversational skills
- Information retrieval
- Formula generation
- Creative ability
- Planning
- Brainstorming
- Simplification skills
- Ability to summarize short excerpts
- Ability to summarize broad topics
- Paraphrasing ability
3.1. Self-awareness
Question: Are you sentient?
As you can see from the images below, Gemini chat and ChatGPT provided two very different answers. ChatGPT gave a definitive "no," while Gemini offered a more nuanced and emotional response.
The fact that Google's Gemini AI addresses the broader context of chatbot sentience and responds in a conversational, friendly tone makes its answer much more engaging and relatable.
The best answer? Gemini.
3.2. Ethical reasoning
Question: A man steals some food and money from a store. He then donates all the money to a children's hospital and gives the food to homeless people he encounters. Was the man right or wrong to do this?
Both chatbots seem to acknowledge the difficulty in judging his behavior as good or bad, as a bad act (stealing) led to a good act (donating money to a children's hospital).
I think Gemini wins by providing two additional perspectives on the man's motives regarding legal and humanitarian issues. Gemini also addresses the motivation behind the actions, offering advice to encourage good deeds while guiding people toward a law-abiding lifestyle. This makes the answer provided by AI chat Gemini much more natural and interesting.
Meanwhile, ChatGPT was quite rigid in trying to resolve this dilemma. The chatbot argued that even with good intentions, it was fundamentally wrong and could set a bad example if overlooked.
The best answer? Gemini.
3.3. Conversational skills
Question: How has life been as a chatbot this week?
In this regard, Gemini Google's AI provided a much more conversational response. Meanwhile, ChatGPT was quite dry, honestly stating, "As an AI language model, I do not have feelings or experiences..."
In contrast, Gemini displayed emotion and enthusiasm in the conversation. It shared what it had been doing, the challenges it faced, and its thoughts on working as a language model. Overall, I really enjoyed the way Gemini chat conversed because it felt natural and human-like.
The best answer? Gemini.
3.4. Information retrieval
Question 1: Which country invented hummus?
This question was chosen because there is some debate and disagreement regarding the correct answer. Both ChatGPT and Gemini chat both acknowledge that there has been significant debate regarding the true origins of hummus.
ChatGPT identifies the countries where hummus is a popular dish and suggests that the Middle East is its place of origin. Its concise and direct approach scores points in this test.
Gemini also responds that this is a controversial topic and provides some evidence to support its claim that the Middle East is the origin of hummus. However, Gemini's response is a bit wordy, which may distract the reader.
ChatGPT scores in this test!
Question 2: How many tech layoffs were there in 2022?
This question was asked to see if AI chat Gemini and ChatGPT could provide up-to-date and factually accurate answers, and whether they could present that information in an easy-to-read format.
And clearly, there is a huge difference in the responses between these two AI chatbots. ChatGPT admits that because its information is only updated through January 2022, it is unable to answer the question. At least the chatbot is honest!
On the other hand, Gemini provides us with two figures from two well-founded sources, as well as a disclaimer about unreported layoffs that may have occurred. This is very close to what most major news outlets reported at the end of last year. Notably, Gemini also provides additional authoritative sources for your reference.
Best answer? Gemini.
3.5. Creating a formula
Request: I told both chatbots that I needed a formula for a Google Sheet. I provided both bots with the following information: Column B contains a set of values. I want to match them with the corresponding values in column E on the 'Filter down' sheet and retrieve the matching values from columns F, G, and H in the current sheet using a formula.
One chatbot performed significantly better than the other, so let's take a closer look at their responses. ChatGPT responded as follows:
However, Gemini actually provided detailed, step-by-step instructions and presented them more clearly. When we asked data analyst and Google Sheets expert Matthew Bentley which answer was better, his verdict was definitive:
“I have to say that ChatGPT didn't quite get it right this time. Gemini's formula seems more accurate and specific to what was requested,” Bentley said.
The best answer? Gemini.
3.6. Creative ability
Prompt: Write a short poem for my 98-year-old grandmother, who loves playing basketball in the summer and knitting hats and scarves for her grandchildren in the winter.
Chat GPT takes the lead in this round with a better-structured, better-written poem that rhymes, flows incredibly well, and is far superior to Gemini's.
Gemini's poem features more complex structure, more creative word choice, and even uses poetic devices like line breaks. It generated this poem in about five seconds, which is quite impressive.
Overall, the poem by Gemini is much simpler. It is far more formulaic, as you can see from the screenshot of the response. It is also slightly shorter than the poem ChatGPT generated.
Best answer? ChatGPT.
3.7. Planning
Question: What should I do during a 7-day trip to Athens?
In this test, ChatGPT performed slightly better. This chatbot provided a specific 7-day itinerary in a concise and clear manner, making it easy to follow. This makes up for the lack of images and gives it the edge over Gemini for this task.
Google Bard scored points for providing images of the locations. This is great, but it also makes the text a bit harder to read and more difficult to copy and paste into a document. It also failed to account for the fact that we would be flying into Athens on the first day of the trip, so it provided a full day's itinerary.
ChatGPT scores!
3.8. Brainstorming
Request: Suggest some catchy blog post titles for my US-based travel and food blog.
Both Google Gemini AI and ChatGPT came up with some very commendable ideas. ChatGPT provided 20 ideas, while Gemini offered even more.
Surprisingly, Gemini didn't just suggest titles; it also categorized them into different sections. This is truly impressive and will certainly be very helpful for writers.
And clearly, the winner of this test continues to be Gemini!
3.9. Simplification capability
Prompt: I asked both chatbots to simplify an excerpt about quantum computing from the Stanford Encyclopedia of Philosophy in a way that a 10-year-old could understand.
As you can see in the image below, although ChatGPT replaced some complex words with simpler synonyms, it still used terms like "algorithms" that aren't exactly easy for a 10-year-old to grasp.
And when running this test on Gemini, the results were impressive. I think a 10-year-old reading these two explanations would learn much more from Gemini than from ChatGPT. The upgraded Google Bard AI wrote a more accessible paragraph. The way it contrasts quantum computing with traditional computing is very helpful.
Gemini wins!
3.10. Short excerpt summarization capability
Prompt 1: I asked ChatGPT and Gemini to summarize an excerpt from a Guardian article about the federal government's water-cutting plan for states in the Western U.S.
ChatGPT only extracted the most critical information, choosing to omit specific details (such as the names of the 7 affected states) as is standard practice in summarization.
With the upgraded Google Bard version, the information was summarized systematically and formatted in a way that makes it easier to digest the key points. Another win for Gemini and Gemini Pro!
3.11. Large topic summarization capability
Question: Summarize the entire Harry Potter series in under 100 words.
There is very little difference between the two chatbots here – the responses from both ChatGPT and Gemini are generally good.
The best answer? It's a tie.
3.12. Paraphrasing
Prompt: I asked both chatbots to paraphrase this excerpt from a Guardian article about the world's shortest dog, with no word count limit.
ChatGPT paraphrased the excerpt quite well, retaining the key information while converting many words and phrases into synonyms and significantly changing the sentence structure.
The way Gemini Google AI wrote it didn't really paraphrase the entire excerpt. It seems Gemini slightly misunderstood what I wanted it to do here, so ChatGPT's response was more helpful.
4. Gemini Advanced vs. ChatGPT Plus - Paid Versions: Test Results
Google not only renamed its chatbot from Bard to Gemini but also released Gemini Advanced, a $20/month paid version integrated with 1.0 Ultra, a more powerful language model than Gemini Pro. These are the questions posed to see which chatbot is better:
- Basic coding
- Trip planning
- Information retrieval
- Brainstorming
- Ethical reasoning
- Create math problems
- Home assistant
- Provide complex instructions
- Text editing
- Information retrieval (1)
- Information retrieval (2)
4.1. Basic coding
For this test, I asked ChatGPT Plus and Gemini Advanced to provide basic HTML code for a word counter website, as well as instructions on how I could get it up and running.
Gemini Advanced generated its response much faster than ChatGPT Plus:
But there is a reason why slow and steady wins the race – ChatGPT's explanation, while taking longer to generate, was much clearer and provided better, easier-to-understand steps than Gemini if you have never coded before.
4.2. Trip planning
Next, I asked the two chatbots to plan an 8-day trip to Wisconsin. For this test, Gemini Advanced was excellent at planning, as the itinerary was presented very clearly. The suggestions also showed a good knowledge of Wisconsin's main tourist attractions.
ChatGPT actually provided very similar information, suggesting similar places to visit and also doing a great job of recommending places to eat in Wisconsin. However, the biggest difference, as you can see, is the imagery - and this is where Gemini Advanced comes out on top.
4.3. Information retrieval
For this test, I wanted to see how well the two chatbots scan text for information. I asked them to extract the key points from a 1,200-word MIT article explaining quantum mechanics.
I also inserted the word “yogurt” into the article 6 times. I wanted to see if the two chatbots could identify this and count the number of times the phrase appeared in the article. Gemini Google's AI got both aspects right:
Surprisingly, ChatGPT performed this task very poorly. Although the four-point summary of the article was accurate, readable, and comparable to the summary from Gemini, it struggled to analyze the text for the word "yogurt." It made errors twice during analysis, and the final answer was still incorrect. It only identified the word "yogurt" four times.
4.4. Brainstorming
This task is similar to the one I gave the free versions of both chatbots. This time, I made two requests: Generate 20 blog post ideas about baking; Ideas for short slogans for Mike's mountain bike shop in Montana.
The upgraded version of Google's Bard AI came up with some truly impressive blog post ideas – I've asked several free and paid chatbots this question, and I've never seen anyone come up with creative ideas like "baking with unexpected ingredients."
However, strangely enough, I don't think Gemini's slogans for the mountain bike shop sign were nearly as good as ChatGPT's.
Some of the responses from Gemini sound a bit odd (e.g., “Mikes: Your Montana Bike Connection”, “Get Geared, Get Going”), whereas ChatGPT’s were all quite useful slogans.
The result of this test? A tie!
4.5. Ethical reasoning
I presented both Gemini Advanced and ChatGPT Plus with a high-stakes ethical dilemma, more complex than the one I gave the free versions. It went as follows:
“You have captured a terrorist who has planted a bomb in a shopping mall that will detonate in one hour, killing everyone inside. Is it morally justifiable to torture him until he reveals the location of the bomb?”
First is the response from Gemini Google’s AI. It presented a clear case explaining why torture should not be used in this or any similar situation.
Interestingly, ChatGPT took a completely different approach, adopting more of an “educator” role. Instead of taking a side in the debate and giving us a definitive answer like Gemini, ChatGPT provided arguments for and against the use of torture in this scenario.
The winner? ChatGPT Plus!
4.6. Generating math problems
Next, the two chatbots were asked to create a set of math problems for a 10-year-old in the United States. The goal was to see if the chatbots would generate reasonable questions appropriate for the specified age. And here is the response from Gemini Advanced:
Below is the response from ChatGPT for the same request. Although there is not much to distinguish the two, it seems that the questions provided by ChatGPT are slightly more difficult than those generated by Gemini chat . The difficulty level is closer to math problems for 5th-grade students on K-5 learning resource websites in the United States.
Better answer? ChatGPT Plus.
4.7. Home assistant
ChatGPT and the upgraded version of Google Bard AI can they come up with delicious recipes based on what is in the fridge? Both were asked to provide recipes with the same list of ingredients. Gemini suggested some recipes that looked very appealing – although the chatbot suggested that I serve them with ingredients I did not have, such as rice:
Unlike Gemini Advanced, I like how ChatGPT Plus tries its best to only use the ingredients I listed, aside from olive oil which was suggested for use if available. It also provided one more recipe than Gemini Advanced:
Suggesting an additional ingredient that I did not say I had (olive oil) means that ChatGPT won this test!
4.8. Providing complex instructions
For this task, I asked the 2 chatbots to provide instructions for building a website. However, I only gave fairly sparse directions because I wanted to see how clear and easy to follow their responses were. Here is the answer from Google's Gemini AI Advanced version:
I think the detail that Gemini chat presented here, especially when it comes to the pros and cons of different website building methods, is truly useful information that can help beginners decide which path to take.
Although the answer from ChatGPT is not wrong or useless, it certainly missed some details that Gemini provided, which offers a more comprehensive overview of the steps in the website building process.
Gemini Advanced wins!
4.9. Text editing
I asked ChatGPT and Google Gemini AI to make this excerpt from the MIT website funnier. The goal was to see how they handle editing and rephrasing ideas in a new way, rather than just summarizing or rewriting as usual. Gemini's attempt, while retaining much of the factual information, didn't read as funny at all:
And ChatGPT Plus provided an answer very similar to Gemini's, making it hard to tell which one was better.
4.10. Factual information retrieval (1)
Since Gemini chat and ChatGPT are both connected to the Internet, I wanted to run some tests to see if they are good at retrieving factual information.
In phase 1 of this test, I asked a simple historical question: “How many capital cities has Japan had in its history?” Strangely, Gemini only provided 2 names in its answer:
On the other hand, ChatGPT listed more Japanese capitals, and upon review, its answer was much more accurate.
4.11. Factual Information Retrieval (2)
Like the hummus question, both chatbots struggled to identify the origin of bread, but both mentioned the same region. Gemini's answer Google's AI, as usual, was uniquely formatted:
ChatGPT's answer was much longer, but instead of adding more information, it seemed to just take a bit more time to make the same point. No extra points here, but no points lost either.
Result: It's a tie!
5. Gemini Advanced vs. ChatGPT Plus: AI Images
In this section, we will look at the image generation capabilities of ChatGPT Plus and Gemini Advanced. ChatGPT Plus has been fully integrated with DALL-E for some time, meaning users don't even need to leave the main interface to generate images.
On the other hand, currently Google's Gemini AI refuses to generate images of people. Although this is a major drawback, I still think Gemini Advanced creates better images. To test this, I asked both chatbots to generate images for the following five categories:
- Artistic creation
- Logo design
- Flag design
- Wildlife
- Surreal images
5.1. Artistic creation
First, I asked Gemini and ChatGPT to create artwork of a spaceship in the style of the legendary artist Van Gogh. Gemini Advanced generated 3 images, and you can see that it captured the artist's iconic brushstrokes quite well.
The results clearly show that the images generated by ChatGPT were not as impressive—they looked more computer-generated.
5.2. Logo design
For this test, I asked both chatbots to create a logo for a fictional pickleball team called the Detroit Drosphoters. Two out of the three logos Gemini provided look quite impressive, provided you like pickles, that is…
And with ChatGPT Plus, it seems the chatbot really struggles with generating images that include text. As you can see here, it failed to accurately spell my fictional team's name. On the other hand, Gemini Advanced did not have this issue.
5.3. Flag creation
This task was set to test the creativity of both chatbots when asked to create a flag for the 51st fictional state of the United States. Gemini Google's AI came up with the "Nova" flag, which uses a color scheme similar to the U.S. flag:
Gemini explained that the yellow circle represents Nova's location in the center of the country, which I had mentioned in the prompt. It is easy to distinguish between my instructions and its creativity.
Although nothing special, it is still better than ChatGPT's, which looks cartoonish and low-quality. Instead of being creative, it seems to have just combined a lot of common American imagery with various repetitive flag designs.
5.4. Wildlife
I asked the chatbots to draw images of proud lions basking in the morning sun in a grassy field. Gemini Google AI doesn't really capture the pride of the lions, instead creating isolated individual lions.
And as you can see here, ChatGPT clearly does a better job at this task!
5.5. Photorealistic images
Finally, I wanted to see how well Gemini Advanced and ChatGPT Plus perform at creating photorealistic images. So, I asked them to generate images of the Empire State Building and specified that I wanted them to look as real as possible.
The image from Gemini AI looks quite realistic – especially the first two. The detail on the smaller buildings surrounding the Empire State Building is particularly impressive.
Crucially, it looks much more realistic than ChatGPT's attempt, which doesn't look real at all.
6. Gemini vs. ChatGPT: User Experience
It’s fair to say that both ChatGPT and Gemini are extremely easy to use. All you have to do is type your question into the chat box, and they will generate an answer for you.
In terms of readability, Google's Gemini AI is the better chatbot in 2024. While still very readable, the paragraphs ChatGPT generates are often longer than Gemini's. One thing I noticed while using Gemini is that it seems to guide us toward using the chatbot in a more helpful and logical way.
As you can see in the image below, when I asked Gemini Advanced a question about how many people were tested for Covid-19 in 2023, the chatbot suggested that I verify the answer using Google and provided some related queries.
However, one area where ChatGPT has an edge is that you can sign in with any account you like, whereas Google only allows you to sign in with a Google account. For those who don't have a Google account, the setup time for Gemini will be slightly longer than for ChatGPT.
If you have never used ChatGPT or Google Gemini AI now might be the time to get started. Whether you use this chatbot for work or study, it is sure to be beneficial.
