Follow-Up Enablement Tools - Concept Note (v0.1)

Objective

Create optional tooling that turns existing call-analysis data into actionable follow-up plans for studio staff, without modifying the deterministic transcription ➜ analysis pipeline. These tools live in the future AgentOrchestrator Lambda and consume the outputs already produced by the core workflow.

Data Sources We Already Have

• call-analysis DynamoDB table – contains follow_up_needed, follow_up_reasons, primary_category, staff_name, call_state, timestamps, etc. (see sample rows shared 2025-09-21).
• staff registry JSON (v1.0) – per-tenant list of staff/aliases; migrates to DynamoDB staff-registry table in v1.1.
• client-configurations DynamoDB table – indicates franchise/site, industry, and configuration versions. AgentOrchestrator can read this to choose the right prompt+schema template when invoking Bedrock tools.

Tool Candidates

1. generate_follow_up_checklist

• Business value: Provide managers with a per-staff checklist of outstanding follow-ups inferred from the past N hours of call analyses.
• Inputs (to AgentOrchestrator):
• tenant_id (franchise#siteId)
• Time window (default: last 24h)
• Optional filter: staff_name
• Pipeline:
• Orchestrator queries call-analysis (GSI by site/time) to fetch items where follow_up_needed = "yes".
• Collapse follow-up reasons using alias-normalized staff names from staff registry.
• Call Bedrock tool with structured prompt that asks LLM to normalize reasons, deduplicate, and assign priorities.
• Tool Spec (Pydantic):

  class FollowUpItem(BaseModel):
      staff_name: str
      customer_name: Optional[str]
      follow_up_reason: Literal[
          "freeze_confirmation", "needs_signature_waiver", "callback_requested_time",
          "billing_issue_unresolved", "custom"
      ]
      details: str
      due_by: datetime
      priority: Literal["high", "medium", "low"]
  
  class FollowUpChecklist(BaseModel):
      generated_for: str  # tenant_id
      generated_at: datetime
      time_window_hours: int
      items: List[FollowUpItem]
      summary: str
  

• Output Usage:
• Persist to s3://{tenant}/follow-ups/YYYY/MM/DD/checklist-{timestamp}.json
• Optional: publish to SNS/email or surface via dashboard API

2. draft_staff_digest

• Business value: Auto-draft daily email/install-notification to each staff member summarizing their calls and pending tasks.
• Inputs:
• tenant_id
• staff_name
• Date (defaults to previous day)
• Pipeline:
• Query call-analysis for staff member and date.
• Summarize call outcomes (success/attempted), highlight follow-ups, revenue priority.
• Call Bedrock tool to draft digest message using reusable email template (prompt module under /templates/{industry}/prompts/tools/daily-digest.txt).
• Tool Spec:

  class StaffDigest(BaseModel):
      staff_name: str
      reporting_date: date
      calls_handled: int
      successes: int
      attempted: int
      follow_ups: List[FollowUpItem]
      highlights: List[str]
      draft_email_subject: str
      draft_email_body: str
  

• Output Usage:
• Return JSON to orchestrator; orchestrator can hand off to SES/SNS (deterministic code) or display in dashboard before sending.

3. generate_staff_performance_report (NEW)

• Business value: Provide individual staff members with detailed performance metrics for self-assessment and improvement, enabling data-driven coaching without manager intervention.
• Inputs:
• tenant_id
• staff_name
• Time period: "daily" or "weekly"
• Date/week (defaults to previous period)
• Pipeline:
• Query call-analysis for staff member over specified period.
• Calculate performance metrics:
  • Call outcomes (success/attempted/na ratios)
  • Revenue impact score (high/medium/low priority calls)
  • Follow-up completion rate (if tracking)
  • Practical coaching items identified
• Compare against:
  • Their previous period performance (trending)
  • Studio averages (benchmarking)
  • Personal best records
• Call Bedrock tool to generate constructive feedback and actionable recommendations.
• Tool Spec:

  class PerformanceMetrics(BaseModel):
      total_calls: int
      success_rate: float  # percentage
      attempted_rate: float
      revenue_impacting_calls: int
      avg_call_duration: Optional[float]
      follow_ups_pending: int
      coaching_areas: List[str]
  
  class PerformanceTrend(BaseModel):
      metric_name: str
      current_value: float
      previous_value: float
      trend: Literal["improving", "declining", "stable"]
      percentage_change: float
  
  class StaffPerformanceReport(BaseModel):
      staff_name: str
      period_type: Literal["daily", "weekly"]
      period_start: date
      period_end: date
      metrics: PerformanceMetrics
      trends: List[PerformanceTrend]
      studio_rank: Optional[int]  # Position among peers
      personal_best: Dict[str, Any]  # Their best metrics to date
      strengths: List[str]  # What they're doing well
      improvement_areas: List[str]  # Specific actionable items
      coaching_recommendations: List[str]  # From AI analysis
      motivational_message: str  # Personalized encouragement
  

• Output Usage:
• Store in s3://{tenant}/performance-reports/{staff_name}/YYYY/MM/DD/report-{period}.json
• Email directly to staff member (private, not shared with managers by default)
• Optional: Display in staff-only dashboard view
• Archive for longitudinal performance tracking

Architectural Placement

• Core pipeline: unchanged. It still writes structured analysis rows.
• AgentOrchestrator:
• Exposes API/command ("Generate checklist", "Daily digest").
• Fetches data from DynamoDB/S3.
• Invokes Bedrock tool with industry-aware prompt (same tool infrastructure introduced in v1.0).
• Validates output via dedicated Pydantic models above.
• Triggers downstream actions (store checklist, present in UI, optional notifications).

Implementation Steps (Incremental)

1. Data Access Layer
2. Add FollowUpRepository in orchestrator to query call-analysis by tenant/time window.
3. Add PerformanceRepository for aggregating staff metrics over periods.
4. Reuse staff alias normalization from staff registry loader.
5. Schema + Tool Definition
6. Add follow_up_models.py with Pydantic models.
7. Add performance_models.py for staff performance tracking.
8. Add tool specs in orchestrator's tool_registry.py (re-using the Bedrock tool pipeline built in v1.0).
9. Create prompt modules under templates/{industry}/prompts/tools/ for checklist + digest + performance instructions.
10. Orchestrator Flow
11. Add new command handlers: GenerateFollowUpChecklistCommand, GenerateStaffDigestCommand, GeneratePerformanceReportCommand.
12. Ensure outputs persist to S3 and optionally queue notifications (SNS/EventBridge).
13. Schedule EventBridge rules for automatic daily/weekly performance report generation.
14. Surface to Users
15. API response returning checklist JSON.
16. Staff-only authenticated endpoints for performance reports.
17. Optional: CloudWatch dashboard counts outstanding follow-ups per staff.
18. Mobile-friendly web view for staff to access their reports.
19. Future Enhancements
20. Integrate with self-service staff registry once DynamoDB migration is complete.
21. Allow managers to confirm/complete follow-up items (stored in separate table follow-up-status).
22. Gamification: badges, streaks, achievements based on performance metrics.
23. Peer comparison opt-in (anonymous rankings).
24. Integration with scheduling systems to correlate performance with shift patterns.

Benefits of Staff Performance Reports

For Individual Staff Members

• Self-Assessment: Private performance metrics without manager oversight pressure
• Continuous Improvement: Daily/weekly feedback loop for rapid skill development
• Recognition: Highlight strengths and celebrate personal bests
• Motivation: Personalized encouragement based on actual performance trends
• Career Development: Track progress over time for performance reviews

For Studio Operations

• Autonomous Improvement: Staff self-correct without manager intervention
• Reduced Coaching Time: AI identifies specific areas for improvement
• Consistent Standards: All staff measured against same metrics
• Early Warning System: Identify declining performance before it impacts revenue
• Team Building: Optional peer comparison for healthy competition

Implementation Considerations

• Privacy: Staff reports are private by default, not visible to managers
• Opt-In Sharing: Staff can choose to share reports with managers for coaching
• Positive Framing: Focus on improvement rather than criticism
• Actionable Feedback: Specific recommendations, not vague suggestions
• Cultural Sensitivity: Customize tone/approach per franchise culture

Open Questions

• Do we want the LLM to assign due dates or should deterministic code derive them from follow_up_reasons rules?
• Should checklists be immutable snapshots (audit) or living tasks that feed back into call-analysis?
• Where should we expose the orchestrator interface first (CLI, internal API, web dashboard)?
• For staff performance reports:
• Should managers have override access to staff reports for HR purposes?
• How long should we retain historical performance data?
• Should we implement peer comparison as opt-in or opt-out?
• What metrics matter most for different industries (fitness vs healthcare vs banking)?

Alignment with Seven Foundations

• Intelligence: Prompt modules for checklist/digest transform raw data into actionable summaries.
• Memory: Uses call-analysis history and staff registry as memory store.
• Tools: Introduces new Bedrock tools invoked by AgentOrchestrator.
• Validation: Structured outputs validated via Pydantic models before use.
• Control: Deterministic orchestrator logic handles data gathering, notification routing.
• Recovery: Checklist generation can fall back to deterministic rules if tool fails (retry ➜ store error).
• Feedback: Future enhancement—allow managers to confirm tasks, feeding back into DynamoDB.