Up to 17% of AI conference reviews now written by AI

Novel statistical analysis reveals significant AI-generated content in recent ML conference peer reviews. What's it mean for scientific integrity?

“Figure 1: Shift in Adjective Frequency in ICLR 2024 Peer Reviews. We find a significant shift in the frequency of certain tokens in ICLR 2024, with adjectives such as “commendable”, “meticulous”, and “intricate” showing 9.8, 34.7, and 11.2-fold increases in probability of occurring in a sentence. We find a similar trend in NeurIPS but not in Nature Por…