IBIA’s Executive Director Alexander Prokopakis joined Intent Communications CEO Namrata Nadkarni to discuss the outcomes of the second round of our Maritime AI comparison project. In this 20 minute discussion, we pull back the curtain on what our judges were thinking when they crowned ChatGPT as the winner, with Grok coming in second place. IBIA members also contributed to this result as they shared their scores on social media with us. You can see the individual scores from our judges and read some of the metrics for scoring here.

Listen to the podcast on spotify by clicking here.

Here is a very quick summary of the podcast discussion:

• Both judges felt that anonymously evaluating the results was beneficial to overcome bias, but each of the LLMs also displayed a bias for the outcomes based on their sources.
• Sources in terms of both number and quality played a large factor in building trust in the outcomes, but completeness and quality were deemed more important than quantity. And some of the LLMs included non-maritime references.
• Rotterdam and Singapore consistently appeared in the top rankings, but the third port ranking flipped between Fujirah or Zhoushan due to LLMs accessing different sources.
• The LLMs demonstrated usefulness in providing overall output, but our judges would not trust them completely without human review and verification of the information. Our judges pointed out that AI cannot replicate the unlisted, nuanced, or confidential conversations that occur between industry leaders, which remain vital for future decision-making.
• Additionally, comments from IBIA members made it clear that non-maritime sources are missing the boat when it comes to accurate assessment of new regions for bunkering hubs.

If you want to get future LLM comparison questions and results delivered straight to your inbox along with other news, expert insights and articles, be sure to subscribe to our monthly newsletter.