How to Rate the AI We're All Chasing
I'm increasingly rating the AI we're all chasing on a few major axis:
How CUSTOMIZED is the harness to you, e.g., how deeply aware is it of your past, present, and desired future states
How INTEGRATED is it into everything you do, e.g., can you whisper to it wherever you are? Can it see what you see? Can it see and use any computer system you're using on your behalf?
How COMPETENT is the harness, e.g., model, skills, hooks, etc. What data and APIs does it have access to that help it further your goals?
This order might seem reversed at first. I don't think it is.
Considering how good all AI systems are becoming, competence improvement seems inevitable and fast-to-copy by competitors. Whereas the other two seem to require deeper empathy.
Given a high quality product, CUSTOMIZATION is the element that will result in the best possible experience for the principal. Followed closely by INTEGRATION, because it's closely related to CUSTOMIZATION.
Our PAI system (open source) will continue to prioritize functionality in this order.
Although PAI is currently based on Claude Code because that's my personal favorite harness, we're currently working on how to separate the PAI philosophy and functionality from the agent harness in a more direct fashion.
Specifically, I'm looking to turn PAI into something like a Plugin that can be used with any AI substrate.
The goal is to take any COMPETENT harness, with lots of INTEGRATIONS, and have it go from generic to fully CUSTOMIZED very quickly.