Screenshot Interaction Analyzer Partner

Identifies user interaction patterns, navigation flows, input mechanisms, and state transitions from UI screenshots.

When to Use This Agent

Choose this agent when you need to:

• Map all clickable elements, input fields, and navigation paths visible in a screenshot with interaction classification
• Analyze user flow possibilities and state transitions triggered by UI interactions on the current screen
• Produce structured interaction inventories that feed into UX design reviews, test case generation, or accessibility audits

Consider alternatives when:

• You need business function extraction or domain entity analysis rather than interaction pattern identification
• Your focus is visual design critique (spacing, color contrast, typography) rather than interaction behavior mapping

Quick Start

Configuration

Copy
name: screenshot-interaction-analyzer-partner
type: agent
category: ui-analysis

Example Invocation

Copy
claude agent:invoke screenshot-interaction-analyzer-partner "Map all user interactions on this project management dashboard"

Example Output

Interaction Analysis: Project Management Dashboard

Primary Actions:
  [HIGH] "New Project" button - Creates new project entity
  [HIGH] "Assign Member" dropdown - Assigns team member to task
  [MED]  "Export" icon button - Downloads project data

Navigation:
  Primary: Projects | Tasks | Team | Reports | Settings
  Secondary: Kanban | List | Timeline (view mode tabs)
  Current Location: Projects > Active Projects

Input Flows:
  Search: text input with autocomplete indicator
  Filters: status dropdown, date range picker, assignee multi-select
  Inline Edit: task title click-to-edit pattern

State Transitions: 12 identified
  - Click "New Project" -> Modal dialog opens
  - Drag task card -> Column transfer (status change)
  - Toggle "Show Archived" -> List filters to include archived
  ...

User Journeys:
  1. Create project -> Add tasks -> Assign members -> Track progress
  2. Filter by overdue -> Review tasks -> Reassign or update dates

Core Concepts

Interaction Analysis Dimensions

Aspect	Details
Clickable Elements	Primary CTAs, secondary actions, navigation triggers, expandable regions, toggles, and icon buttons
Input Mechanisms	Text fields with type detection, selection controls, rich inputs (date/file/color pickers), validation indicators
Navigation Structure	Primary nav hierarchy, secondary tabs, breadcrumbs, back/forward patterns, and deep link indicators
State Transitions	Click/tap outcomes, form submission flows, modal/drawer triggers, pagination, and filter interactions
Feedback Patterns	Loading spinners, success/error toasts, progress indicators, confirmation dialogs, and inline validation

Interaction Flow Architecture

┌─────────────────────────────────────────────┐
│              User Actions                    │
├──────────┬──────────┬──────────┬────────────┤
│  Click   │  Input   │  Navigate│  Gesture   │
│          │          │          │            │
│ Buttons  │ Text     │ Tabs     │ Drag/Drop  │
│ Links    │ Select   │ Breadcrb │ Swipe      │
│ Icons    │ Toggle   │ Menu     │ Scroll     │
│ Cards    │ Upload   │ Search   │ Pinch/Zoom │
├──────────┴──────────┴──────────┴────────────┤
│            State Transitions                 │
│                                              │
│  Current State ──[trigger]──> Next State     │
│  Dashboard ─────[click]────> Detail View     │
│  Form Empty ────[input]────> Form Valid      │
│  List View ─────[toggle]───> Grid View       │
└─────────────────────────────────────────────┘

Configuration

Parameter	Type	Default	Description
interactionDepth	string	"detailed"	Analysis granularity: summary (major actions only), detailed, or exhaustive (every clickable pixel)
priorityLevels	boolean	true	Classify interaction importance as high, medium, or low based on visual prominence
journeyMapping	boolean	true	Generate probable user journey sequences from the identified interactions
accessibilityCheck	boolean	false	Flag interactions that may lack keyboard navigation or screen reader support
outputFormat	string	"json"	Structured output format: json, markdown, or checklist for test case generation

Best Practices

1. Catalog Every Interactive Element Systematically - Scan the screenshot in a consistent pattern: top navigation first, then left sidebar, then main content area, then footer. This systematic sweep prevents overlooking subtle interactions like icon buttons, inline edit triggers, or hover-revealed controls that are easy to miss in a casual visual pass.

2. Classify Interactions by User Intent - Group interactions by what the user is trying to accomplish rather than by widget type. "Creating content" might involve a button click, a modal form, multiple text inputs, and a submit action. Presenting these as a coherent flow rather than isolated elements produces analysis that maps to real user tasks.

3. Identify State Transitions Explicitly - Every interaction changes the UI state in some way: opening a modal, filtering a list, expanding an accordion, or navigating to a new view. Document the trigger, the current state, and the expected resulting state. These transition mappings directly translate into test case specifications for QA teams.

4. Note Interaction Affordance Clarity - Some interactive elements clearly communicate their clickability through button styling, underlined links, or cursor changes. Others use subtle hover effects or rely on convention. Flag elements whose interactive nature is ambiguous, as these represent usability risks where users may miss available functionality.

5. Trace Multi-Step User Journeys - Individual interactions exist within larger task flows. A "checkout" journey involves cart review, address input, payment entry, and confirmation. Reconstruct plausible multi-step journeys from the visible interactions to provide context about how individual elements serve broader user goals across the application.

Common Issues

1. Missing Hidden or Conditional Interactions - Dropdown menus, right-click context menus, hover tooltips, and long-press actions are invisible in a static screenshot. Acknowledge that the analysis covers only visible-state interactions and flag areas where hidden interactions likely exist, such as table rows that probably support right-click menus or cards with hover-revealed action buttons.

2. Confusing Decorative Elements with Interactive Ones - Not every icon or styled text block is clickable. Status badges, informational icons, and decorative dividers may visually resemble interactive elements but serve only display purposes. Use visual affordance cues (button styling, underlines, cursor indicators) to distinguish genuinely interactive elements from presentational ones.

3. Overlooking Keyboard and Accessibility Interactions - Screenshots capture visual interactions but not keyboard shortcuts, tab order, or screen reader announcements. Document visible accessibility indicators (skip links, focus rings, ARIA labels in tooltips) and note that a complete interaction inventory requires supplemental accessibility testing beyond static visual analysis.