Each month, we are posing questions about the maritime industry to the public versions of five leading AI platforms – ChatGPT, Claude, Copilot, Gemini and Grok, before inviting an expert guest judge to help us assess which LLM made the grade. 

We’ve written previously about our criteria for scoring the LLMs – and how we assess the accuracy, completeness and ease of understanding offered by the answers, alongside a final overall impression score. Each of these four sections were scored out of 10 to provide an overall mark out of 40 – we’ve included the judges’ final scores at the end of this page. 

For our first edition, we asked: Why wasn’t the Net Zero Framework adopted at the recent IMO MEPC Extraordinary Session?

You can read the outputs from the individual LLMs by clicking here.

Judging the results

Intent Communications CEO Namrata Nadkarni joined Contributing Editor Ed Parshotam were two of the judges and we invited Stuart Neil, former Director of Strategy and Communications at the International Chamber of Shipping, to join us as our expert judge. 

Each judge scored the LLMs independently of each other and – for our first and second place, at least – we came to a unanimous decision: Gemini was our winner, with Copilot second. You can see the detailed scores at the bottom of this article.

With the scoring complete, Namrata and Ed discussed the results – and identified a number of areas where LLMs either overdelivered or underperformed. You can watch the discussion on Spotify here.

Questionable sources

Most of the LLMs provided something of a mixed bag when it came to the reliability of sources they drew upon to provide their responses. Grok, in particular, was notable, receiving positive marks as the only LLM to rely on the IMO website for its answer, while also being the only platform in our test to reference Facebook and LinkedIn posts, which felt less authoritative. Meanwhile, Claude was marked down for its reliance on a single source – the law firm Vinson & Elkins – which would naturally limit its ability to provide a rounded perspective. 

Missing nuance and detail

A number of the LLMs failed to capture the nuance of what unfolded at the MEPC Extraordinary Session.  ChatGPT, in particular, seemed to downplay the hostility towards the adoption of the Net Zero Framework from the US and Saudi Arabia. Claude and ChatGPT also failed to provide a timeline illustrating the steps to the Net Zero Framework being considered for adoption in October 2026. 

First and second place

Copilot made it far clearer than ChatGPT, for example, that a major element in the Net Zero Framework not being adopted was the opposition of Member States such as the United States and Saudi Arabia. However, while its answer was clear and more closely reflected this nuance, it did not use the IMO or any shipping bodies for its source material. 

The unanimous verdict of the judges was that Copilot was edged into second place by Gemini, which offered an easy-to-follow answer that detailed the reasons for opposition (including the economic impact and political pressure), and pointed out that technical work is still progressing despite the one-year adjournment.

The missing human element 

We should give the final word to our expert judge, who neatly encapsulated one of the main limitations of AI, noting via email that all of the LLM’s answers had failed to convey “the emotion or tension in the room”. Because Stuart was in the room, it makes perfect sense that he’d be able to supply insight that we’d anticipate an AI solution would miss.

More broadly, humans will always have the edge over machines when it comes to emotive issues and human behaviour. This first edition of our AI comparisons has only served to underline that LLMs can play a useful role but must always be paired with human oversight and an understanding of their limitations. 

Final scores (out of 40):

Stuart Neil, former Director of Strategy and Communications at the International Chamber of Shipping: ChatGPT – 26; Claude – 23; Copilot – 26; Gemini – 30; Grok – 24. 

Namrata Nadkarni, CEO, Intent Communications: ChatGPT – 20; Claude – 20; Copilot – 26; Gemini – 29; Grok – 22. 

Ed Parshotam, Senior Editor, Intent Communications: ChatGPT – 24; Claude – 24.5; Copilot – 27; Gemini – 28; Grok – 25.