With the rapid release of new AI models, it can be difficult to determine which performs best for real-world tasks. We built an automated benchmark to test them.

The prompt was simple but demanding. We provided zero architectural constraints beyond requiring React, Tailwind, and a mock authentication flow. We wanted to see how the models structure a project from scratch when left entirely to their own devices.

The Methodology

We ran the exact same zero-shot prompt through the API of both models. Temperature was set to 0.2 to reduce variance while allowing slight creative freedom in UI design.

• Prompt Length: ~450 words
• Key Requirements: Context API for state, Tailwind for styling, mocked JWT auth flow, responsive dashboard layout.
• Evaluation metric: Pass/Fail on compiling without errors, followed by a manual UX heuristic review.

"GPT-4o gave us a polished UI but hallucinated a dependency. Claude 3.5 Sonnet gave us a functional UI, and the code worked flawlessly on the first paste."

GPT-4o: Polished UI, but Hallucinated Dependencies

GPT-4o writes code that looks visually impressive. Its Tailwind usage is sophisticated, utilizing complex grid layouts and modern utility combinations. However, it made a critical error in state management.

import { createContext, useContext, useReducer } from 'react';
import { fakeAuthApi } from '@mock/api'; // ERROR: Hallucinated import

const AuthContext = createContext(null);

export const AuthProvider = ({ children }) => {
  const [state, dispatch] = useReducer(authReducer, initialState);

  const login = async (email, password) => {
    const user = await fakeAuthApi.login(email, password);
    dispatch({ type: 'LOGIN_SUCCESS', payload: user });
  };
  // ... rest of the context

Notice line 2. It assumed the existence of a @mock/api package instead of building the mock function inline as requested. This caused an immediate build failure.

Claude 3.5 Sonnet: Self-Contained and Functional

Claude took a different approach. It built the mock infrastructure inline, ensuring the file was self-contained and runnable immediately. It properly isolated concerns and even added JSDoc comments to the mock functions.

While its UI was slightly more basic (relying heavily on standard flexboxes rather than advanced CSS grids), it scored a perfect 1.0 on our strict compilation test.