Evaluation
Wizard of Oz User Testing
The team conducted Wizard of Oz testing using a paper prototype. One team member acted as the system, manually switching screens whenever a participant tapped a button, while participants thought aloud throughout the session.
Participants
Four college students from two different INFO 360 group — matching the target user profile of students who regularly use Canvas, apps, and online tools.
Task Scenarios
Results & Findings
✓ What Worked Well
Group 2 understood that the design was helping them make a privacy decision before accepting cookies. The “Medium Risk” was easy to notice, and they also knew that the tool was warning them about possible privacy concerns. The dashboard idea also made sense because it showed where saved websites or privacy choices could be stored for later review.
✗ Problems & Confusion
Group 2 was not sure whether the prototype was an app, a website pop-up, or a browser extension. They also understood the “Medium Risk”, but they wanted to know why the risk was medium. Another confusion was the difference between “Reject All” and “Apply Safe Settings Automatically.” The participant was also less likely to notice the “See Full Summary” button because it did not stand out enough.
→ Key Learnings
Students want quick privacy guidance, but they still need enough explanation to trust the tool’s recommendation. The risk level is useful, but it should include a short reason, like what data is being collected or shared. The safest action should be visually clear, and each button should explain what will happen if the user chooses it. We also learned that the product needs a clear label, like “Browser Extension,” so users immediately understand what type of tool they are using.
