Avoid the AI Expertise Trap

The further away a topic is from your expertise, the smarter an AI will sound.

This is a blind spot that not enough people are aware of or take seriously.

The best possible way to test the expertise of an AI system is to have it perform analysis or work in an area where you're a pinnacle expert.

Stepping even a few degrees off from your center-mass expertise will make it sound smarter. And the further you go, the smarter it will sound.

I recommend you have a series of deep questions within your area of expertise, where you know the arguments backwards and forwards.

Using those as part of your core benchmarks for testing new AI will be a lot more useful than the feeling of throwing random tasks at it.

Anyone with sufficient language skills can sound smart in areas you know little about.