Home – Cycle 1 – Cycle 2 – Cycle 3 – Cycle 4 – Cycle 5 – Cycle 6
Cycle 1 – Understand It – Experiment: Watch the Prediction Change
How to Use This Page
- Run the guided experiment with the instructor.
- Notice how the model’s explanation changes when the information changes.
- Then choose one independent scenario below.
- Run the starting prompt first.
- Add each fact one at a time and compare the results.
Guided Experiment
What we’re testing: An LLM generates plausible explanations from learned patterns. Change the information it receives, and different patterns become more or less relevant.
Example 1 – Interpreting a Customer Situation
Step 1: Start with very little information.
A customer who has purchased from us for three years suddenly stops ordering. What are some likely reasons?
Step 2: Change the situation.
A customer who has purchased from us for three years suddenly stops ordering. Their industry is experiencing a significant downturn. What are some likely reasons?
Step 3: Change it again.
A customer who has purchased from us for three years suddenly stops ordering. Their industry is growing rapidly and they recently hired several new employees. What are some likely reasons?
Example 2 – Interpreting a Production Problem
Step 1: Start with very little information.
A production line that normally meets its daily target has fallen 15% below target this week. What are some likely causes?
Step 2: Add information that changes what is plausible.
A production line that normally meets its daily target has fallen 15% below target this week. Production volume increased significantly this week. What are some likely causes?
Step 3: Change the evidence again.
A production line that normally meets its daily target has fallen 15% below target this week. Production volume has not changed, but several experienced operators were replaced by new employees. What are some likely causes?
Compare / Reflect
- Did the model’s first answer sound reasonable?
- How did the explanations change as the information changed?
- Which possibilities became more or less likely?
- Did the model actually know what happened, or was it generating plausible explanations from learned patterns?
Independent Experiment
Choose one scenario below.
- Run the starting prompt and review the model’s possible explanations.
- Add Fact 1 to the starting prompt and run it again.
- Add Fact 2 as well and run it again.
- Add Fact 3 and run it one final time.
- Compare all four responses. Notice what changed as the available information changed.
Scenario 1 – Sales Are Declining
Starting Prompt:
Sales have declined 12% over the last three months. What are some likely causes?
Add these facts one at a time:
- Fact 1: Website traffic and lead volume have remained relatively stable.
- Fact 2: The percentage of qualified prospects receiving proposals has also remained stable.
- Fact 3: The percentage of proposals that become sales has declined substantially.
Watch for: Does the model move away from explanations about awareness and lead generation and toward explanations involving later stages of the sales process?
Scenario 2 – Production Output Is Falling
Starting Prompt:
A production line that normally meets its daily target has been producing 15% below target for the past month. What are some likely causes?
Add these facts one at a time:
- Fact 1: Customer demand and scheduled production volume have not changed.
- Fact 2: Equipment downtime and material availability are approximately the same as before the decline.
- Fact 3: Several experienced operators recently moved to another shift and were replaced by newer employees.
Watch for: Which early explanations become less relevant as each new fact is introduced?
Scenario 3 – Employee Turnover Is Increasing
Starting Prompt:
Employee turnover has increased significantly over the past six months. What are some likely causes?
Add these facts one at a time:
- Fact 1: Pay and benefits have not changed during that period.
- Fact 2: Most of the increase is occurring among employees with less than one year of service.
- Fact 3: The company significantly increased hiring six to nine months ago and shortened its normal onboarding and training process.
Watch for: Does a broad set of possible explanations become increasingly focused as more information is added?
Scenario 4 – Use Your Own Situation
Task: Apply the same experiment to a real situation from your work.
Choose something where you are trying to understand why something is happening. This could involve sales, production, quality, customers, employees, projects, costs, scheduling, or another issue relevant to you.
- Start simple. Describe only the basic situation and ask the AI what some likely explanations might be.
- Review its response. Notice how many possibilities it generates from limited information.
- Add one fact you know that could help narrow the possibilities. Ask the question again.
- Add another relevant fact and run it again.
- Continue once or twice more if useful.
- Compare the responses.
Pay attention to:
- Which possibilities disappeared as you added information?
- Which became more likely?
- Did new possibilities appear?
- What did the AI actually know, and what was it predicting based on learned patterns?
CIRAS (Center for Industrial Research And Service, ISU) • IIAI (Iowa Initiative for Artificial Intelligence, UofI) • TrAC (Translational Artificial Intelligence Center, ISU)
 |
 |
 |
This course was developed through a collaborative effort to advance AI literacy across organizations in Iowa.