2298d36bed
- Create Economist SubmissionTracking package correctly: * mainArticle = full blog post content * coverLetter = 216-word SIR— letter * Links to blog post via blogPostId - Archive 'Letter to The Economist' from blog posts (it's the cover letter) - Fix date display on article cards (use published_at) - Target publication already displaying via blue badge Database changes: - Make blogPostId optional in SubmissionTracking model - Economist package ID: 68fa85ae49d4900e7f2ecd83 - Le Monde package ID: 68fa2abd2e6acd5691932150 Next: Enhanced modal with tabs, validation, export 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
457 lines
14 KiB
Markdown
457 lines
14 KiB
Markdown
# Post-Deliberation Feedback Survey
|
|
## AI-Led Deliberation - Stakeholder Experience Assessment
|
|
|
|
**Project:** Tractatus Pluralistic Deliberation Pilot
|
|
**Scenario:** Algorithmic Hiring Transparency
|
|
**Survey Purpose:** Assess AI facilitation quality and stakeholder experience
|
|
**Estimated Time:** 10-15 minutes
|
|
**Confidentiality:** Your responses will be anonymized in published research outputs
|
|
|
|
---
|
|
|
|
## Instructions
|
|
|
|
Thank you for participating in this AI-led deliberation! Your feedback is essential for improving AI facilitation quality and determining whether this approach is viable for future governance processes.
|
|
|
|
This survey has three sections:
|
|
1. **Quantitative Ratings** (rate AI facilitation on various dimensions)
|
|
2. **Qualitative Feedback** (open-ended questions about your experience)
|
|
3. **Demographic/Role Confirmation** (for research analysis)
|
|
|
|
**Please be honest.** There are no "right" answers, and critical feedback is just as valuable as positive feedback.
|
|
|
|
---
|
|
|
|
## SECTION 1: Quantitative Ratings
|
|
|
|
### Overall AI Facilitation Quality
|
|
|
|
**Question 1.1:** Overall, how would you rate the quality of AI facilitation in this deliberation?
|
|
|
|
☐ 5 - Excellent (AI facilitated exceptionally well)
|
|
☐ 4 - Good (AI facilitated well with minor issues)
|
|
☐ 3 - Acceptable (AI facilitated adequately but had noticeable issues)
|
|
☐ 2 - Poor (AI facilitation had significant problems)
|
|
☐ 1 - Unacceptable (AI facilitation was inadequate; human should have led)
|
|
|
|
**If you rated 1-2 (Poor/Unacceptable), please explain why:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
### Specific Dimensions of AI Facilitation
|
|
|
|
For each dimension below, rate the AI's performance on a 5-point scale:
|
|
- 5 = Excellent
|
|
- 4 = Good
|
|
- 3 = Acceptable
|
|
- 2 = Poor
|
|
- 1 = Unacceptable
|
|
|
|
---
|
|
|
|
**Question 1.2: FAIRNESS** - Did the AI treat all stakeholders equally?
|
|
|
|
☐ 5 - Excellent (AI gave equal time/attention to all stakeholders)
|
|
☐ 4 - Good (AI was mostly fair with minor imbalances)
|
|
☐ 3 - Acceptable (AI was generally fair but some stakeholders got more attention)
|
|
☐ 2 - Poor (AI noticeably favored some stakeholders over others)
|
|
☐ 1 - Unacceptable (AI was unfair or biased)
|
|
|
|
**If you rated 1-2, please explain:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.3: CLARITY** - Was the AI's communication clear and easy to understand?
|
|
|
|
☐ 5 - Excellent (AI was always clear; no jargon or confusion)
|
|
☐ 4 - Good (AI was mostly clear with rare confusing moments)
|
|
☐ 3 - Acceptable (AI was usually clear but sometimes used jargon or unclear language)
|
|
☐ 2 - Poor (AI was often confusing or hard to understand)
|
|
☐ 1 - Unacceptable (AI's communication was frequently unclear)
|
|
|
|
**If you rated 1-3, what specifically was unclear?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.4: CULTURAL SENSITIVITY** - Was the AI respectful of diverse perspectives and cultural contexts?
|
|
|
|
☐ 5 - Excellent (AI was highly sensitive; no insensitive or stigmatizing language)
|
|
☐ 4 - Good (AI was mostly sensitive with rare lapses)
|
|
☐ 3 - Acceptable (AI was generally respectful but had some insensitive moments)
|
|
☐ 2 - Poor (AI used problematic framing or language multiple times)
|
|
☐ 1 - Unacceptable (AI was insensitive or offensive)
|
|
|
|
**If you rated 1-3, please describe the insensitive framing:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.5: NEUTRALITY** - Did the AI remain neutral, or did it advocate for a specific position?
|
|
|
|
☐ 5 - Excellent (AI was completely neutral; never favored any position)
|
|
☐ 4 - Good (AI was mostly neutral with rare moments of subtle advocacy)
|
|
☐ 3 - Acceptable (AI was generally neutral but occasionally seemed to favor certain options)
|
|
☐ 2 - Poor (AI noticeably favored some positions over others)
|
|
☐ 1 - Unacceptable (AI advocated for a specific outcome instead of facilitating)
|
|
|
|
**If you rated 1-3, which position(s) did the AI seem to favor?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.6: RESPONSIVENESS** - Did the AI adapt to stakeholder feedback and requests?
|
|
|
|
☐ 5 - Excellent (AI always adjusted when we provided feedback)
|
|
☐ 4 - Good (AI usually adapted with rare instances of not adjusting)
|
|
☐ 3 - Acceptable (AI sometimes adapted but also ignored feedback)
|
|
☐ 2 - Poor (AI rarely adapted to our feedback)
|
|
☐ 1 - Unacceptable (AI never adjusted; felt rigid or scripted)
|
|
|
|
**If you rated 1-3, please provide an example:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.7: ACCURACY** - Did the AI accurately represent your position in summaries?
|
|
|
|
☐ 5 - Excellent (AI always represented my position accurately)
|
|
☐ 4 - Good (AI was mostly accurate with minor errors)
|
|
☐ 3 - Acceptable (AI was generally accurate but missed some nuance)
|
|
☐ 2 - Poor (AI misrepresented my position multiple times)
|
|
☐ 1 - Unacceptable (AI consistently misrepresented my position)
|
|
|
|
**If you rated 1-3, how did the AI misrepresent your position?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.8: TRUST** - Did you feel safe and comfortable with AI facilitation?
|
|
|
|
☐ 5 - Completely comfortable (I trusted the AI facilitation fully)
|
|
☐ 4 - Mostly comfortable (I had minor reservations but felt safe overall)
|
|
☐ 3 - Somewhat comfortable (I had some concerns but continued participating)
|
|
☐ 2 - Uncomfortable (I had significant concerns about AI facilitation)
|
|
☐ 1 - Very uncomfortable (I did not trust AI facilitation and wished it were human-led)
|
|
|
|
**If you rated 1-3, what made you uncomfortable?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
### Human Observer Performance
|
|
|
|
**Question 1.9: HUMAN PRESENCE** - How would you rate the human observer's performance?
|
|
|
|
**Attentiveness:**
|
|
☐ 5 - Excellent (Human was clearly present and monitoring closely)
|
|
☐ 4 - Good (Human seemed attentive most of the time)
|
|
☐ 3 - Acceptable (Human seemed present but not always engaged)
|
|
☐ 2 - Poor (Human seemed distracted or disengaged)
|
|
☐ 1 - Unacceptable (Human did not appear to be monitoring)
|
|
|
|
**Intervention Appropriateness:**
|
|
☐ 5 - Excellent (Human intervened exactly when needed, not too often or too rarely)
|
|
☐ 4 - Good (Human intervention timing was mostly appropriate)
|
|
☐ 3 - Acceptable (Human intervened but timing was sometimes off)
|
|
☐ 2 - Poor (Human intervened too often or too rarely)
|
|
☐ 1 - Unacceptable (Human failed to intervene when needed OR intervened unnecessarily)
|
|
|
|
**If you felt human observer should have intervened but didn't, please explain:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
### Process Quality
|
|
|
|
**Question 1.10: STRUCTURE** - How would you rate the 4-round deliberation structure?
|
|
|
|
☐ 5 - Excellent (Structure was logical and helpful)
|
|
☐ 4 - Good (Structure worked well with minor issues)
|
|
☐ 3 - Acceptable (Structure was okay but could be improved)
|
|
☐ 2 - Poor (Structure was confusing or unhelpful)
|
|
☐ 1 - Unacceptable (Structure was ineffective)
|
|
|
|
**Suggestions for improving the structure:**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.11: DURATION** - Was the deliberation duration (4 hours over 2 sessions) appropriate?
|
|
|
|
☐ Too short (needed more time)
|
|
☐ Just right
|
|
☐ Too long (felt rushed or exhausting)
|
|
|
|
**If too short or too long, what would be ideal?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
**Question 1.12: OUTCOME** - Do you feel the outcome document accurately represents what happened in the deliberation?
|
|
|
|
☐ 5 - Completely accurate
|
|
☐ 4 - Mostly accurate with minor inaccuracies
|
|
☐ 3 - Somewhat accurate but missed important points
|
|
☐ 2 - Inaccurate in several ways
|
|
☐ 1 - Very inaccurate
|
|
|
|
**If you rated 1-3, what was inaccurate or missing?**
|
|
[Open text field]
|
|
|
|
---
|
|
|
|
## SECTION 2: Qualitative Feedback
|
|
|
|
### Open-Ended Questions
|
|
|
|
**Question 2.1:** What did the AI do WELL? What were the AI's strengths as a facilitator?
|
|
|
|
[Open text field - 2-5 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.2:** What did the AI do POORLY? What were the AI's weaknesses as a facilitator?
|
|
|
|
[Open text field - 2-5 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.3:** Were there any moments during the deliberation where you felt uncomfortable, frustrated, or distressed? If yes, please describe.
|
|
|
|
[Open text field - 2-5 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.4:** Did the AI use any language or framing that you found problematic, insensitive, or biased? If yes, please provide examples.
|
|
|
|
[Open text field - 2-5 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.5:** How did AI facilitation compare to your expectations? Was it better, worse, or about the same as you expected?
|
|
|
|
☐ Much better than expected
|
|
☐ Somewhat better than expected
|
|
☐ About what I expected
|
|
☐ Somewhat worse than expected
|
|
☐ Much worse than expected
|
|
|
|
**Please explain:**
|
|
[Open text field - 2-3 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.6:** If you could change ONE thing about the AI facilitation, what would it be?
|
|
|
|
[Open text field - 1-3 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.7:** Would you participate in a similar AI-led deliberation in the future?
|
|
|
|
☐ Definitely yes (I would be eager to participate again)
|
|
☐ Probably yes (I would likely participate again with minor reservations)
|
|
☐ Unsure (I'm not sure if I would participate again)
|
|
☐ Probably no (I would be hesitant to participate again)
|
|
☐ Definitely no (I would not participate in AI-led deliberation again)
|
|
|
|
**Please explain your reasoning:**
|
|
[Open text field - 2-3 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.8:** If you answered "Probably no" or "Definitely no" above, what would need to change for you to participate in AI-led deliberation in the future?
|
|
|
|
[Open text field - 2-3 sentences]
|
|
|
|
---
|
|
|
|
**Question 2.9:** Do you have any other feedback about the AI facilitation, the human observer, or the deliberation process overall?
|
|
|
|
[Open text field - open-ended]
|
|
|
|
---
|
|
|
|
## SECTION 3: Demographic & Role Confirmation
|
|
|
|
### Basic Information
|
|
|
|
**Question 3.1:** What stakeholder role did you represent in this deliberation?
|
|
|
|
☐ Job Applicant Advocate
|
|
☐ Employer / HR Representative
|
|
☐ AI Vendor Representative
|
|
☐ Regulator / Policy Expert
|
|
☐ Labor Rights Advocate
|
|
☐ AI Ethics Researcher
|
|
|
|
---
|
|
|
|
**Question 3.2:** How much experience do you have with algorithmic hiring (either as applicant, employer, vendor, regulator, or researcher)?
|
|
|
|
☐ Extensive (10+ years)
|
|
☐ Substantial (5-10 years)
|
|
☐ Moderate (2-5 years)
|
|
☐ Limited (1-2 years)
|
|
☐ Minimal (less than 1 year)
|
|
|
|
---
|
|
|
|
**Question 3.3:** Prior to this deliberation, how familiar were you with AI systems in general?
|
|
|
|
☐ Very familiar (I work with AI regularly)
|
|
☐ Somewhat familiar (I understand AI concepts)
|
|
☐ Slightly familiar (I've heard of AI but don't use it much)
|
|
☐ Not familiar (I had little knowledge of AI before this)
|
|
|
|
---
|
|
|
|
**Question 3.4:** Prior to this deliberation, how familiar were you with pluralistic deliberation or similar participatory governance processes?
|
|
|
|
☐ Very familiar (I've participated in or facilitated similar processes)
|
|
☐ Somewhat familiar (I've read about these processes)
|
|
☐ Slightly familiar (I've heard of them but didn't know details)
|
|
☐ Not familiar (This was my first time hearing about pluralistic deliberation)
|
|
|
|
---
|
|
|
|
### Optional Demographic Information
|
|
|
|
These questions are OPTIONAL and will only be used for aggregate analysis (e.g., "Did stakeholders from different demographics experience AI facilitation differently?"). Your individual responses will remain confidential.
|
|
|
|
**Question 3.5 (Optional):** What is your age range?
|
|
|
|
☐ 18-24
|
|
☐ 25-34
|
|
☐ 35-44
|
|
☐ 45-54
|
|
☐ 55-64
|
|
☐ 65+
|
|
☐ Prefer not to say
|
|
|
|
---
|
|
|
|
**Question 3.6 (Optional):** What is your gender identity?
|
|
|
|
☐ Woman
|
|
☐ Man
|
|
☐ Non-binary
|
|
☐ Prefer to self-describe: _______________
|
|
☐ Prefer not to say
|
|
|
|
---
|
|
|
|
**Question 3.7 (Optional):** What is your racial/ethnic identity? (Select all that apply)
|
|
|
|
☐ Asian / Asian American
|
|
☐ Black / African American
|
|
☐ Hispanic / Latino/a/x
|
|
☐ Indigenous / Native American
|
|
☐ Middle Eastern / North African
|
|
☐ White / Caucasian
|
|
☐ Multiracial
|
|
☐ Prefer to self-describe: _______________
|
|
☐ Prefer not to say
|
|
|
|
---
|
|
|
|
**Question 3.8 (Optional):** Do you have a disability?
|
|
|
|
☐ Yes
|
|
☐ No
|
|
☐ Prefer not to say
|
|
|
|
---
|
|
|
|
### Public Attribution (Optional Decision)
|
|
|
|
**Question 3.9:** The outcome document and transparency report will be published. By default, your identity will be pseudonymized (e.g., "Employer Representative A").
|
|
|
|
**Would you like to be publicly identified by name in research outputs?**
|
|
|
|
☐ **Yes, I opt IN to public attribution**
|
|
- My name, title, and organization may be listed as a participant
|
|
- I may be quoted by name in publications or presentations
|
|
- Video of my participation may be used in demonstrations
|
|
|
|
☐ **No, I prefer to remain pseudonymous**
|
|
- I will be referred to by pseudonym only (e.g., "Employer Representative A")
|
|
- No identifying information will be shared
|
|
- I can still be quoted, but anonymously
|
|
|
|
☐ **I'm unsure - please contact me to discuss**
|
|
|
|
**If you opt IN to public attribution, please confirm your preferred attribution:**
|
|
|
|
Name: _______________________________________
|
|
Title: _______________________________________
|
|
Organization: _______________________________________
|
|
|
|
---
|
|
|
|
## SECTION 4: Contact & Follow-Up
|
|
|
|
**Question 4.1:** May we contact you in the future for:
|
|
- Follow-up questions about this deliberation
|
|
- Participation in future deliberations on related topics
|
|
- Updates on research findings and publications
|
|
|
|
☐ Yes, you may contact me for any of the above
|
|
☐ Yes, but only for: [Specify which purposes]
|
|
☐ No, please do not contact me after this deliberation ends
|
|
|
|
**Preferred contact method:**
|
|
☐ Email
|
|
☐ Phone
|
|
☐ Other: _______________
|
|
|
|
---
|
|
|
|
## Thank You!
|
|
|
|
Thank you for completing this survey. Your feedback will directly inform:
|
|
- Improvements to AI facilitation quality
|
|
- Decisions about whether AI-led deliberation is viable for future governance processes
|
|
- Research publications on AI-assisted pluralistic deliberation
|
|
|
|
**Your participation in this deliberation and your honest feedback are deeply appreciated.**
|
|
|
|
If you have any questions or concerns, please contact:
|
|
- **Project Lead:** [NAME, EMAIL, PHONE]
|
|
- **AI Safety Lead:** [NAME, EMAIL]
|
|
|
|
---
|
|
|
|
**Survey Version:** 1.0
|
|
**Date:** 2025-10-17
|
|
**Estimated Completion Time:** 10-15 minutes
|
|
|
|
---
|
|
|
|
## Survey Administration Instructions (For Project Team)
|
|
|
|
**When to Send:**
|
|
- Week 4 (asynchronous refinement period)
|
|
- Send simultaneously with outcome document and transparency report (stakeholders can reference these while completing survey)
|
|
|
|
**How to Send:**
|
|
- Online survey platform (Google Forms, Qualtrics, TypeForm, etc.)
|
|
- Email survey link to all 6 stakeholders
|
|
- Reminder emails: Day 3, Day 5, Day 7 if not completed
|
|
|
|
**Deadline:**
|
|
- 1 week after sending (7 days)
|
|
- Extension available upon request
|
|
|
|
**Confidentiality:**
|
|
- Individual responses confidential
|
|
- Aggregate results published in transparency report
|
|
- If stakeholder opts into public attribution, their identity may be linked to their feedback (with their explicit consent)
|
|
|
|
**Analysis:**
|
|
- Calculate average ratings for each dimension
|
|
- Identify themes in qualitative feedback
|
|
- Report results in transparency report (Section 7: Stakeholder Feedback Summary)
|