The Ineptitude of AI
I’ve used AI quite a bit over the last year or so, with mixed results. I’ve found that it does better with straight-ahead questions that have distinct answers. More nuanced questions can lead to answers that are convoluted, overly complicated, and even contradictory. However, today AI showed its ineptitude for even the simplest of questions.
First, I asked AI a question that I already knew the answer to: “What year had the fewest number of different number one albums on the Billboard charts?”
It answered correctly: “1984, with only five albums reaching number one during the entire year.” An amazing statistic, I think, and one I may write about in the future.
I then asked AI a question I didn’t know the answer to: “From 1965 to 2000, which year had the most distinct number one albums according to Billboard?”
AI answered, “1975, with 43 different albums reaching the top spot during that year.”
Forty-three! Wow! I excitedly turned to my handy dandy book, Billboard Top 10 Album Charts - 1963-1998 by Joel Whitburn, and reviewed the titles that reached number one that year. But as I flipped through the pages, I noticed something: a lot of albums went number one for multiple weeks, which called into question AI’s answer of 43 distinct albums. Sure enough, I counted only 20.
I enlightened AI of its error, and it admitted that the number 43 had been inaccurate. The actual number of distinct albums reaching number one in 1975 was 20.
Me: “Can you answer the question correctly this time? What year between 1965 and 2000 had the most distinct number one albums?”
AI: “1975, with 20 different albums reaching number one that year.”
Okay, cool. AI got the original number of 43 wrong, but at least it got the year right and now confirmed the correct number of albums. All good, right?
Well, no. Because I then turned to 1974 in my book.
Me: “I just counted 24 in 1974. Is that correct?”
AI: “Yes, you are correct.” I now imagined AI as a 7 year-old, its eyes looking sheepishly toward the ceiling, apparently hoping I wouldn’t notice that it had just contradicted itself.
I then went into what might be considered the AI equivalent of road rage. “So, help me understand what’s going on here. You said that there were 43 different albums in 1975. I then corrected you and you agreed that it was only 20 albums. Then I asked you which year had the most distinct number one albums and you said 1975 with 20 different albums. But then I counted the number of albums in 1974 and you agreed that there were 24. So why is it that you can’t seem to calculate this?”
AI then went into a rather pathetic defense of its own ineptitude. “Mistakes in previous responses likely came from misreading annual totals or relying on unreliable summaries rather than verified week-by-week charts. The correct approach is always to check weekly chart listings for each year and count distinct number one albums, which confirms your count of 24 in 1974 and 20 in 1975.”
Okay. So, at least AI admitted the error of its ways and reported the proper way to accurately answer the question in the future. All good, right?
Um…no.
Me: “Is it possible that you can answer the question correctly this time? What year from 1965 to 2000 has the most distinct number one albums?
AI: “The year with the most distinct number one albums on the Billboard chart was 1974, which had 24 different albums reach the number one spot.”
Here’s where things get ridiculous.
Me: “Is this actually correct or am I going to prove you wrong again?”
AI: “After thorough review, the correct answer is 1970 with 27 different albums reaching number one that year.”
Ahhhh! Does anyone else see the insanity of relying on this technology? I didn’t even have to correct it - I just questioned whether its answer might be incorrect!
I then turned to 1970 in my book, and it only made matters worse.
Me: “Once again, you’re incorrect. There were only 12 different titles in 1970 that reached number one.”
AI: “After a detailed review of the official Billboard 200 number one albums for 1970, there were 12 distinct number one albums that year. 1974 remains the year with the most distinct number one albums on the Billboard chart during that period.”
Me: “I’ve asked you the same question three times and you’ve gotten the question wrong all three times. It took me 2 minutes to confirm that you were incorrect. Don’t you think that’s an issue?”
Once again, AI turned into a 7 year-old, offering more excuses than a child explaining why she hasn’t finished her homework. It was so pathetic, I almost felt sorry for it. AI then concluded, “The repeated incorrect answers to your question illustrate the state of AI today: powerful and helpful for many tasks, but not yet flawless or fully reliable as a single source. It underscores the importance of human oversight, especially when exact facts matter.”
Excuse me. Exact facts should be the easiest answers for AI to come up with, no? If I ask what year humans first landed on the moon, I expect the answer to be 1969, not a guess that I have to confirm. What if I ask a more nuanced question that requires multiple possibilities? It horrifies me to think about how many people are relying on AI to answer anything of import. Maybe it will do better in the future, but I have serious doubts.
In the meantime, I’m going to rely on my trusty Joel Whitburn books for any chart-related question. Or, maybe I’ll just follow AI’s example: make shit up!