A research study analyzing the collaborative writing process between humans and generative AI tools in short story creation.
How does co-working with AI affect the fiction-writing process, and the fiction-rating process?
Upload a story you wrote with any degree of LLM or multi-modal AI assistance
Help identify which parts were written by humans, AI, or through collaboration
Rate stories on creativity, structure, and artistic achievement
The study takes part in three phrases:
Writers submit completed short flash fiction stories, alongside complete chat logs or interaction histories with AI tools.
Volunteer data taggers examine each story and the provided chat links to identify and categorize text snippets as purely human-authored, AI-generated, or collaboratively produced.
Volunteers also categorize the key elements of each story - characters, plot elements, and literary devices.
In parallel, an LLM extracts the key elements of the story.
Volunteer readers evaluate stories on their literary merit and enjoyability, including narrative structure, character development, and plot points.
Then, the stories also undergo a mixed human-AI rating process, using techniques from Google DeepMind Rater Assist. Finally, an AI will generate an independent rating of the story's literary merit and technical execution.
We're analyzing how writers integrate AI assistance into their creative workflow and decision-making processes1. By comparing stories with different collaboration patterns, from minimal AI assistance to extensive co-creation, we aim to understand what new workflows are emerging.2.
We also aim to understand how both human- and AI-perceived quality vary with the degree and type of AI assistance or collaboration. What patterns can we observe?
When given free rein to use AI assistance, what parts of a story do writers use assistance on? What is the difference between what an AI tags as important in a story, versus what humans do? How do human ratings of a story change when they are prompted by an AI on what qualities to rate on?
This research contributes to understanding how AI tools affect creative quality, authorship attribution3, and the future of collaborative writing. Results will inform both academic research and practical guidelines for writers working with AI assistance4.