ChatGPT4 Passes The Humanity Test

 

 

In the Coda to my book, “Man Verses Machine”, I thought it only fair to ask ChatGPT - by now on Version 4 - to write a song of its own. After all, it had spent several weeks in my company, discussing and analysing over 50 of my songs. It had certainly demonstrated that it had a grasp on the form of a poem or a song lyric, but one question remained open: did it have anything to say.

Obviously I believed it did not. It takes a lifetime of experience, feeling both joy and grief, living with hope and bereavement, and developing the talent to express all this in a way that touches other people’s hearts and minds.

I thought it might be impolite to say this to ChatGPT - we’ve all read the stories about its close relative, Bing, and its hurt feelings. I needn’t have worried though. CHatGPT was frank and realistic, ironically demonstrating a self-awareness and self-critical honesty rare in most poets. It said:

“As an AI language model, I am capable of generating text, including lyrics and song structures, based on prompts provided to me. However, my ability to write songs is limited by my programming and training data, and my output may not always reflect the emotional depth and nuance that human songwriters are capable of conveying.”

That is its belief and that is my own belief about its creative powers.

Then it wrote its song, which you can find in the book. It is not a great song, but in all honesty, it isn’t terrible either. I have heard worse burping from the speakers of pop radio. However, so much has been made about the (real) dangers of AI-generated content infiltrating not only the creative arts but also academia and news media that the fear has spawned several online tools claiming to be able to identify it.

I have no insight into how this AI-sniffing algorithm works, but like other writers I’m unsettled by the thought that it might one day misidentify my own writing - born out of the highs of ecstasy and the lows of mental breakdown - as the work of a machine. But that aside, I have startling news!

GhatGPT’s lyrics passed all three AI-identifying systems with flying colours. Two of the three were 100% certain its song was human-generated, and the third was 97% positive.

In short, it passed the test. According to Writer, AI Writing Check from Quill, and Content At Scale, three of the top systems, ChatGPT-4’s work was indistinguishable from that of a human.

Whether this will green-light its use in song-factories remains to be seen. I wouldn’t be against it. Many pop songs today are dashed out quickly by a committee of a dozen writers. They are so far removed from the authentic life-experiences of a single writer or partnership that they may as well have been written by a word-aggregator - which is essentially what an AI is. When words are merely sounds to fill spaces between ‘beats’, it is unlikely that any tool will be off the table when it comes to spewing them out with industrial efficiency.

I believe there will one day be a new “authenticity” movement which will demand that the music and lyrics it consumes are from a human source, communication heart to heart, soul to soul. But one shouldn’t get too smug about it, or too aloof. There are many songs in the pop cannon which literate songwriters might turn their noses up at, but which are loaded with meaning for someone else - songs which may have a single line which speaks to someone or songs that remind a person of a time or place, or indeed, another person who is important to them. For regular folks, songs are more than just words.

To play Devil’s Advocate for a moment, if we are asking whether a text was generated by an AI or a human rather than whether it has the power to move, inspire, or comfort someone, perhaps we are asking the wrong question.

But if the question is a simple: “Can ChatGPT-4 pass as human?” The answer is yes, yes it can.