How Do You Judge AI-Enabled Creative Work? A Guide to Evaluating Creative AI Outputs

Megan Workmon Larsen
5 min readOct 21, 2024

--

As someone entrenched in both the worlds of creativity and technology, I’ve witnessed firsthand the tension generative AI brings to the creative landscape. Is it a revolutionary collaborator or a threat to human artistry? The debates rage on, but I see it differently: AI isn’t here to steal our creative spark — it’s here to challenge us, to force us to rethink what creativity means in the evolving digital age. Yet with this potential comes responsibility, new risks, and new solutions, like model collapse, where AI models degrade by training on their own outputs, and data poisoning, where artists fight back by corrupting AI’s training data to protect their work​. How do we ensure AI-generated outputs aren’t just efficient, but ethical, inclusive, and truly innovative? This evaluation rubric cuts through the hype, offering a framework to critically assess AI’s contributions without losing sight of the human element that makes art, well, art. AI might be powerful, but it’s still a tool — it’s up to us to guide it toward enhancing our creative process, not replacing it.

A human mess (Adobe Firefly)

This rubric is designed to help creators, critics, and evaluators navigate the complexities of AI-generated creative work. It offers a structured way to assess outputs based on multiple criteria — ranging from ethics and cultural inclusivity to aesthetic value and emotional impact. By breaking down these key dimensions, the rubric provides a holistic view of how AI can enhance or detract from creative projects.

Here’s the first draft of the rubric I’ve developed to help guide these evaluations:

Criteria Overview

Ethics and Attribution

Evaluates the alignment of AI-generated output with ethical standards, including intellectual property respect and clear attribution of any sources used in the creative process.

Cultural Representation and Inclusivity

Assesses whether AI output respects and promotes diverse cultural expressions, ensuring marginalized voices are represented without oversimplification or bias.

Aesthetic and Artistic Contribution

Examines the visual, auditory, or experiential quality of AI output, considering how well it enhances the overall creative experience without overshadowing human input.

Emotional and Experiential Impact

Measures the emotional depth and connection fostered by AI-generated work and how well it contributes to a meaningful artistic experience.

Reliability and Trust

Looks at the consistency and dependability of the AI output, considering whether it delivers high-quality results that build trust in its creative capabilities.

Workforce Impact

Considers the broader effect of AI on job creation, skill development, and potential displacement within the creative industries.

Creative Autonomy and Control

Evaluates the balance between AI involvement and human creative freedom, ensuring that the artist retains control over the final output.

Evaluating Generative AI Tools for Creative Work

While the rubric focuses on evaluating AI-generated creative outputs, it’s just as important to critically assess the tools behind them. When choosing an AI tool for your creative projects, both practical and ethical considerations must come into play. Below are key criteria and guiding questions to ensure that your chosen tool aligns with your vision, values, and workflow. This approach ensures you make informed decisions about integrating AI, so it enhances — not compromises — your creative process.

1. Aesthetic Contribution

  • Does the AI tool create innovative, compelling, or emotionally engaging outputs that align with your creative goals?
  • How customizable is the tool in terms of style, tone, and aesthetic choices? Can it be adjusted to match your vision?
  • Are you satisfied with the level of creative depth and originality it provides?

2. Emotional and Experiential Impact

  • Can this AI tool generate content that evokes the emotional responses and immersive experiences you want for your audience?
  • How well does the tool understand audience engagement, and can it personalize emotional resonance?

3. Human-AI Collaboration

  • Will this AI tool enhance your creative process or risk taking over key decision-making aspects?
  • Does the tool allow for creative interventions and adjustments, or does it limit your ability to guide the process?
  • Does it offer meaningful suggestions that complement your creativity, or do you feel a lack of control over its outputs?

4. Creative Autonomy and Control

  • Do you retain full control over the final output, or does the tool limit your creative autonomy?
  • Can you easily override or modify the AI’s suggestions to ensure alignment with your artistic vision?
  • Does the tool provide enough flexibility to take creative risks, or does it constrain experimentation?

5. Ethics and Attribution

  • Does the AI tool respect intellectual property rights, and are there clear guidelines on how it sources data?
  • Is the tool transparent about its datasets and how it was trained?
  • Will you be able to clearly and ethically attribute the contributions of the AI?

6. Cultural Representation and Inclusivity

  • How does the AI tool handle cultural diversity? Are there safeguards to avoid bias and misrepresentation?
  • Has the AI been trained on diverse datasets, or does it risk reinforcing dominant cultural narratives?
  • Can the tool authentically represent different cultural perspectives without oversimplification?

7. Long-Term Use and Integration

  • How well does the tool integrate with other creative tools and workflows you use?
  • Will using this tool help you evolve creatively in the long term, supporting your goals?
  • Are you comfortable with the potential impact this tool may have on jobs in the creative field?

Shake it Up

As AI continues to shake up the creative world, it’s not just about embracing shiny new tools — it’s about critically evaluating how they’re shaping what we create. This rubric is just the starting point, and I’m excited to see how you apply it. Does this framework resonate with your experience? Did I miss anything, or is there something you’d tweak? I’d love to hear your thoughts, whether you’re a creator, technologist, or somewhere in between. Let’s keep the conversation going and refine this tool together — so as AI and human creativity collide, they enhance each other, not compete. What do you think? Let’s chat!

--

--

Megan Workmon Larsen
Megan Workmon Larsen

Written by Megan Workmon Larsen

Rebellious educational researcher, storyteller, and artist with an operatic flair and human-centered approach. Teaching AI now, because why not?

No responses yet