Need help?

For questions, assistance, or to report an issue, please contact the COLTT Help Desk at 956-665-5327 or 956-882-6792.

Browse Help Articles Submit a Ticket

Quick Links

Evaluating AI-Generated Responses

When using generative A.I. for content creation, the subsequent step involves critically examining its responses after requesting the A.I. to perform a task. It is essential to recognize that AI-generated content is not infallible—it may contain errors or omissions. As users, our responsibility lies in verifying the accuracy of the information provided. On this platform, we will explore techniques to assess the quality of the A.I.'s work and ensure reliable information.

R.A.C.C.C.A. Framework: Modified Version

The R.A.C.C.C.A. Framework is a structured guide for responsible development, deployment, and use of generative A.I. technologies. ACUE. (2023) has taken the acronym and tailored it to help evaluate the responses provided by generative A.I. Let us break it down:

  1. Relevance: Does the response directly address the prompt? Like asking a question that connects with what you are learning, something happening in the world, or a real problem you want to solve.
  2. Accuracy: Is the response factually correct? Refrain from accepting the response given; the user must verify that AI-generated responses are factually correct and reliable.
    1. Example: After generating an essay on historical events, cross-reference the AI-generated dates, figures, and events with credible sources to ensure factual accuracy.
  3. Completeness: Does the A.I.'s response include all elements requested in prompt? If you asked for a few things, did the gen AI response address all items requested?
  4. Clarity: Is the A.I.'s answer easy to understand? If it is too complicated, consider asking the generative A.I. to simplify the response more concisely.
  5. Coherence: Does the A.I.'s answer make sense from start to finish? The response generated should flow nicely and be organized. Read through the response; if unorganized, prompt the generative A.I. to organize the response flow beginning with ... [suggest a better order].
  6. Appropriateness: Is the answer right for who is listening or reading? It should fit the situation, whether for school, work, or just for fun, and be respectful and suitable for the audience. Remember, mastering this modified framework empowers you to use A.I. effectively and responsibly. If the initial answer needs to be corrected, tweak your question and try again.

References

ACUE (2023). AI Quick Study Series.
OpenAI (2024). ChatGPT4. [Large Language Model]. 

Contributors

  • Dr. Jessica Sanchez, Center for Online Learning & Teaching Technology
  • Dr. Randall Monty, Writing & Language Studies
  • Raymundo Garza, Center for Online Learning & Teaching Technology