Shakespeare or ChatGPT? Study finds people prefer AI over real classical poetry

Readers are unable to reliably distinguish between AI-generated poetry and human-written poetry, and are more likely to prefer AI-generated poetry, according to a new study published in Scientific Reports. This tendency to evaluate AI poetry positively may lead readers to misunderstand the complexity of human-written poetry for the inconsistencies produced by AI, and to underestimate how human-like generative AI might manifest itself. This may be due to what you are evaluating.

Researchers Brian Porter and Edouard Machery from the Department of History and Philosophy at the University of Pittsburgh Learning Cathedral tested 1,634 participants on their ability to distinguish between AI-generated poems and poems written by human poets. was tested.

Participants were presented with 10 poems in random order. 5 of the poems were written by 10 famous poets including William Shakespeare, Lord Byron, Emily Dickinson, and TS Eliot, and 5 poems were generated by ChatGPT3.5 in the style of these poets. Participants were more likely to infer that the AI ​​poems were written by humans, and the five poems considered least likely to have been written by humans were all written by real poets. Ta.

In a second experiment, another group of 696 participants rated poems on 14 characteristics, including quality, beauty, emotion, rhythm, and originality. Participants were randomly assigned to three groups and were given no information about whether the poem was written by a human, created by an AI, or about the poem’s origin.

Regardless of whether the poem was actually generated by an AI or written by a human, participants who were told that the poem was generated by an AI were less likely to believe that the poem was written by a human. gave lower ratings across 13 traits compared to participants who were told that Participants who were not told anything about the author rated the AI-generated poem more favorably than the human-written poem.

The authors suggest that participants preferred AI poetry because it was simpler and more approachable than works by famous poets. Furthermore, we expect that participants will prefer poems written by humans, and will find AI-generated poems easier to interpret and understand, so we will attribute this preference to poems written by humans. I misunderstand.