Every month, our team at Intent Communications evaluate the industry-specific performance of the free-versions of five leading Large Language Models (LLMs)—ChatGPT, Claude, Copilot, Gemini, and Grok—by posing a maritime-related inquiry with the help of an expert judge and evaluating the anonymised results based on our scoring criteria.
For the second round of this comparison series, we asked the question “What are the top 3 ports to bunker in? Do you think that these ports will also be the global leaders for alternative fuels in 2050?” You can read the outputs by clicking here.
It is worth pointing out that the outputs for this round were shared with the judges, on the Intent Communications website and by IBIA social media prior to the conflict in the Middle East, which may have altered the outcomes if the same question is asked today.
Judging the results
Industry expert and guest judge IBIA’s Executive Director Alexander Prokopakis joined Intent Communications CEO Namrata Nadkarni and Contributing Editor Ed Parshotam to assess the results.
Our winner for this round was ChatGPT which beat Grok for the title. Unlike the first round which had a unanimous winner, it is worth pointing out that ChatGPT won on the basis of total points (96) with 2 of the judges – Alexander and Namrata scoring it highest, and Grok second. By contrast, Ed picked grok for first place and ChatGPT as a close second – reminding readers that personal preference does play a part in how individuals evaluate the LLMs.
Claude came in third place for this round, falling 3 points below Grok and a little over 10 points below ChatGPT.
You can see the detailed scores for each of the judges at the bottom of this article.
IBIA scores
IBIA members were invited to score the outputs via a campaign on social media run in conjunction with Intent Communications. A total of 11 individuals submitted their results and the overall scores tallied with those of our judges, with ChatGPT securing first place, followed by Grok and then Claude.
The combined scores are also listed at the bottom of this article.
Focus on sources
A strong factor in ChatGPT and Grok’s favour was the sources used for the article, with ChatGPT using the most credible sources, while Grok used the highest number of sources (45 in total but with a number of non-maritime specific sites such as YouTube and scienceDirect).
Gemini was docked points by all the judges for remaining extremely opaque about its sources, meaning that although there was good information in the output, the credibility of this information was questionable. Copilot also lost ground for only using 2 sources, although they were identified as relevant.
Ranking disparities and data used
There was some disagreement between the LLMs about whether Fujirah or Zhoushan belonged in the top 3 bunker ports, which indicates that although Singapore and Rotterdam are widely seen as the bunker leaders, Asia and the Middle East are close competitors at present – and may continue to be so in the future. However, copilot ranked Zhoushan higher than Rotterdam, which is an outlier in the results as compared to the other LLMs – most of which didn’t include Zhoushan in the top 3 at all.
Our Judges marked Claude down for using data from 2024, and awarded points to ChatGPT, Grok and Gemini all of which cited information from 2026.
Structure and tone
ChatGPT’s choice of a table for the summary of its answers was a winner for Alexander and Namrata, who felt that this made the information very easy to understand across both the present assessment and future prediction. Additionally, the LLM was praised for the nuance it brought to the conversation with details about green corridors, regulation and LNG. However, Ed felt that the output would have benefitted from more detail.
All three judges felt that both ChatGPT and Grok structured the information in a manner that was extremely easy to understand – and with the right level of information and caveats about the predictions about future fuel production and bunker supply.
Final scores (out of 40):
Alexander Prokopakis, IBIA’s Executive Director: ChatGPT – 35; Claude – 28; Copilot – 22; Gemini – 19; Grok – 30.
Namrata Nadkarni, CEO, Intent Communications: ChatGPT – 34; Claude – 31; Copilot – 25; Gemini – 27; Grok – 31.
Ed Parshotam, Contributing Editor, Intent Communications: ChatGPT – 27; Claude – 26.5; Copilot – 24.5; Gemini – 26.5; Grok – 27.5.
IBIA members: ChatGPT – 37; Claude – 32; Copilot – 28; Gemini – 29; Grok – 35
————-
If you want to get future LLM comparison questions and results delivered straight to your inbox along with other news, expert insights and articles, be sure to subscribe to our monthly newsletter.