Which AI chatbot is best for coding?

ChatGPT artificial intelligence chat bot It represents the presence of many users, but recently, the new robot under the title Gemini He has entered competitions and has the ability to code. In this article, we answer the important and controversial question of which AI chatbot performs better for coding.

If you’re stuck on a programming project, you might be looking for a tool to help you brainstorm ideas, write clean code, or explain a complex concept. Which AI chatbot is better for coding in your opinion, and which one would you choose: the fast and informative Gemina, or the comprehensive and powerful ChatGPT? Next, we introduce the best artificial intelligence chat bot for coding.

Language support

When it comes to language support, ChatGPT surpasses Gemina in terms of breadth and skill. While Gemina officially supports around 22 popular programming languages ​​– including Python, Go and TypeScript – ChatGPT’s language capabilities are much broader.

Unlike Gemini, ChatGPT does not have an official list of supported languages. However, the bot can handle not only the popular languages ​​Gemina supports, but dozens of others, from newer ones like TypeScript and Go to older ones like Fortran, Pascal, and BASIC.

To test their language capabilities, I tried simple coding tasks in languages ​​such as PHP, JavaScript, BASIC, and C++. Both Gemini and ChatGPT performed well with common languages, but only ChatGPT could convincingly put together programs in older languages ​​like BASIC.

Code accuracy and quality

You’re behind on your project deadline and you need a whole lot of code. You ask ChatGPT and Gemina to generate code to implement that functionality, and both tools spew out dozens of lines of code. It seems that with a simple trick, you came out victorious from this challenging situation.

But which tool’s code can you trust to deliver the functionality you’ve requested? To compare the accuracy and quality of code produced by two AI chatbots, I performed a simple coding task on them. I asked Gemini and ChatGPT to create a simple to-do list app using HTML, CSS, and JavaScript. I did not provide any primers. The goal is to see how well both chatbots can work with limited information. ChatGPT (GPT-4o) produced “good enough” functional code. Using the ChatGPT code, you can add or remove a task. Here’s what I got after running the ChatGPT result in the browser:

Then, I asked Gemina from Google to repeat the same thing. Gemini is also able to create a To-do list application was You can also add and remove tasks, but the overall design was not that attractive:

I did the second test. This time I asked both chatbots to recreate the Twitter feed (X.com). ChatGPT produced an old-style Twitter feed with functional tweeting capabilities. I can type in a textbox, send a tweet and dynamically load on the page. It wasn’t the Twitter feed I was hoping for, but given that most of ChatGPT’s training data is full of old Twitter code, the results seem understandable.

Unfortunately, in this round, Google Gemina was unable to provide functional code. It generated hundreds of lines of JavaScript code, but there were too many placeholders to fill in with missing logic. If you’re in a hurry, such code with locations won’t be very useful, as it still requires heavy development work. In such cases, it is more efficient to write the code from scratch.

I tried a few other basic coding tasks and in all cases, the ChatGPT solution was clearly the better option.

4% 150,000

143,260 Toman

Debugging and error detection

Bugs and bugs are like puzzles that programmers have every right to hate! These bugs will drive you crazy, but fixing them is quite satisfying. So when you run into a bug in your code, should you ask Gemina or ChatGPT for help? It really depends on the type of error you want to avoid.

To decide, I gave both AI chatbots two problems to solve. First, I asked both chatbots to solve a logic error in simple PHP code. Logical errors are much harder to detect than syntax errors because it depends on the purpose of the code:

The code in this screenshot is executed and even produces the correct result in many cases. However, it has several logical errors that are not immediately apparent. Can you spot them? I asked Gemina for help and unfortunately the chatbot could not detect the logic error in the code:

None of Gemina’s three attempts to solve the problem were accurate. I tried a similar problem six months ago with the same disappointing result. It seems that Gemina has not progressed in this field.

I then asked ChatGPT for help and it immediately recognized the logic error.

Gemina also rewrote the code to fix this error:

After doing some more troubleshooting, ChatGPT was clearly better at this. However, Gemina’s performance was not entirely in vain. The bot could handle most of the intentional errors I threw at it, but it struggled with complex errors, especially logic errors.

Awareness of context

One of the biggest challenges of using artificial intelligence chatbots for coding is their relatively limited knowledge of the context or topic. They may be able to create separate code for well-defined tasks, but struggle to create the code base for a larger project.

For example, let’s say you’re building a web app with an AI chatbot. You tell it to write code for your signup and login HTML page, and it does just fine. Then ask the chatbot to create a server-side script to handle the login logic. This is a simple task, but due to limited context awareness, it may produce a login script with new variables and naming conventions that do not match the rest of the code.

Which AI chatbot for coding is better at maintaining context awareness? I gave both tools the same programming task: a chat application that we know ChatGPT can already build.

Since the introduction of GPT-4 Turbo and its 128k context window, ChatGPT’s ability to retain much more text, for longer periods of time, has improved significantly. When I first built a chat application with ChatGPT using a 4k GPT-4 context window, it ran smoothly with only minor incidents of text deviation.

A November 2023 reconstruction of the same project with 128k GPT-4 Turbo showed a significant improvement in context awareness. Six months later, in May 2024, there was no significant change in context awareness, but neither was it worse.

Unfortunately, when I first tried Gemina (then called Bard) in the same project, it lost the project path and failed to complete the program. Several rounds of updates later, I tested Gemini again in the same project, and it seems to have gotten worse. So, once again, in terms of context awareness, ChatGPT wins.

problem solving

At this point, Google’s Gemini is lacking in many ways. But can he win in the end? Let’s test its problem-solving abilities. Sometimes you just have a problem, but you’re not sure how to show it programmatically, let alone how to solve it.

In this situation, chat bots like Gemina and ChatGPT can be useful. I asked them both to “write a JavaScript code that counts the number of times a particular word appears in a text”.

In the following, we see the result of Google Gemina:

And then we see the result of ChatGPT:

At first, both approaches seem very solid. Gemina’s approach seems even brief. However, the ChatGPT code has a more robust and accurate approach to counting the number of words in a text. It takes into account word boundaries and case sensitivity, handles punctuation correctly and provides more reliable results. Once again, ChatGPT is superior.

The ChatGPT approach breaks the input text into words in such a way that it can handle all non-word characters such as punctuation marks and special characters as word separators. In the meantime, Gemina considers only the white space as a separator. This approach may fail if the text contains punctuation or other non-word characters in the words, or if the words are not separated by whitespace characters.

Since the Gemina lost in almost every benchmark I used for comparison, I decided to give it a chance at redemption. “ChatGPT vs. Gemini: Which AI Chatbot is Better for Coding?”, I asked the chatbot. We see the result:

Sounds like something I agree with to an extent! I asked ChatGPT what he thought of the assessment and he agreed:

Now, while this sounds perfectly normal, there’s an interesting twist here. For most of the last year, Geminai (then Bard) always confidently stated that he could produce better code, was more efficient, and made fewer mistakes. Here is a screenshot from one of my tests in November 2023:

It seems that Gemina has become a little self-conscious and humble now!

Programming features

Neither ChatGPT nor Gemini has any major features that are exclusive to programming. However, both chatbots have features that can significantly enhance your programming experience if you know how to use them effectively.

ChatGPT offers a set of features that can simplify the programming process when using a chatbot. Useful add-ons like Memory and Custom GPT allow you to customize ChatGPT for your specific programming needs.

For example, the Custom GPT feature can help you create specialized mini versions of ChatGPT for specific projects by uploading the relevant files. This makes things like debugging code, optimizations, and adding new features much easier. Overall, ChatGPT has more features than Google Gemina that can improve your programming experience.

ChatGPT is in a league of its own!

Google’s Gemini has been heavily hyped, so it might be surprising to see how much it lacks compared to ChatGPT. While ChatGPT was clearly off to a good start, you might think Google’s massive resources would help it take that advantage away. Despite these results, it would be unwise to dismiss Gemini as a programming aid. Although Geminai is not as powerful as ChatGPT, Geminai bot still has impressive performance and is evolving at a fast pace.

Source: Makeuseof