feat: Add cloud mode for PR review workflow

[xingyaoww]

Summary

This PR adds a new 'cloud' mode to the PR review workflow that launches reviews in OpenHands Cloud instead of running locally in GitHub Actions. This is inspired by and builds upon PR #1927.

Changes

• examples/03_github_workflows/02_pr_review/agent_script.py:

  • Add MODE environment variable support (sdk or cloud)
  • Add post_github_comment() helper function to post PR comments
  • Add _start_cloud_conversation() helper function to call OpenHands Cloud API
  • Add run_cloud_mode() function that creates a cloud conversation and posts a comment with the URL
  • Refactored existing logic into run_sdk_mode() function
  • Cloud mode only requires OPENHANDS_CLOUD_API_KEY (ignores LLM_MODEL and LLM_BASE_URL)

• .github/actions/pr-review/action.yml:

  • Add new inputs: mode, openhands-cloud-api-key, openhands-cloud-api-url
  • Update validation step to check correct secrets for each mode
  • Update run step to pass new environment variables

• examples/03_github_workflows/02_pr_review/workflow.yml:

  • Add mode input with comment explaining both modes
  • Add commented cloud mode secrets section for easy configuration

• examples/03_github_workflows/02_pr_review/README.md:

  • Add "Two Review Modes" feature description
  • Updated setup documentation with separate sections for SDK and Cloud mode
  • Add Cloud Mode Benefits section
  • Update Action Inputs table with new cloud mode inputs

• tests/github_workflows/test_pr_review_agent.py (new):

  • Add 14 tests covering cloud mode functionality

Cloud Mode Benefits

• Faster CI completion: Exits immediately after starting the review
• Track progress in UI: Users can follow the review in OpenHands Cloud
• Interactive: Users can interact with the review conversation
• Uses cloud model: The LLM model is configured in the OpenHands Cloud account, not in the workflow

Usage

SDK Mode (default):

- name: Run PR Review
  uses: OpenHands/software-agent-sdk/.github/actions/pr-review@main
  with:
      mode: sdk
      llm-api-key: ${{ secrets.LLM_API_KEY }}
      github-token: ${{ secrets.GITHUB_TOKEN }}

Cloud Mode:

- name: Run PR Review
  uses: OpenHands/software-agent-sdk/.github/actions/pr-review@main
  with:
      mode: cloud
      openhands-cloud-api-key: ${{ secrets.OPENHANDS_CLOUD_API_KEY }}
      github-token: ${{ secrets.GITHUB_TOKEN }}

Checklist

• If the PR is changing/adding functionality, are there tests to reflect this?
• If there is an example, have you run the example to make sure that it works?
• If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
• If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
• Is the github CI passing?

@xingyaoww can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: http://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	eclipse-temurin:17-jdk	Link
python	amd64, arm64	nikolaik/python-nodejs:python3.12-nodejs22	Link
golang	amd64, arm64	golang:1.21-bookworm	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:9143041-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-9143041-python \
  ghcr.io/openhands/agent-server:9143041-python

All tags pushed for this build

ghcr.io/openhands/agent-server:9143041-golang-amd64
ghcr.io/openhands/agent-server:9143041-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:9143041-golang-arm64
ghcr.io/openhands/agent-server:9143041-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:9143041-java-amd64
ghcr.io/openhands/agent-server:9143041-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:9143041-java-arm64
ghcr.io/openhands/agent-server:9143041-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:9143041-python-amd64
ghcr.io/openhands/agent-server:9143041-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:9143041-python-arm64
ghcr.io/openhands/agent-server:9143041-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:9143041-golang
ghcr.io/openhands/agent-server:9143041-java
ghcr.io/openhands/agent-server:9143041-python

About Multi-Architecture Support

• Each variant tag (e.g., 9143041-python) is a multi-arch manifest supporting both amd64 and arm64
• Docker automatically pulls the correct architecture for your platform
• Individual architecture tags (e.g., 9143041-python-amd64) are also available if needed

[xingyaoww]

🧪 Cloud Mode Test Results (Bug Fix Verification)

Status: ✅ Bug fix verified - Cloud mode working correctly

Issue Found and Fixed

The _start_cloud_conversation function was attempting to pass a secrets field to the OpenHands Cloud API, but this field is not supported by the API schema (causes HTTP 422 "Extra inputs are not permitted" error).

Fix Applied

• Removed the secrets field from the API request payload
• Added informational log message explaining that OpenHands Cloud uses connected GitHub accounts instead

Test Results

• API Key: ✅ Valid and working
• Conversation Creation with github_token: ✅ Successful (no more 422 error)
• Conversation ID: 5be1faf2ef364d3f84df3055922867db

📍 Test Conversation URL: https://app.all-hands.dev/conversations/5be1faf2ef364d3f84df3055922867db

This test was performed to verify the bug fix for PR #1966.

[xingyaoww]

🔧 Major Refactor: Cloud Mode Now Uses OpenHandsCloudWorkspace

I've refactored the cloud mode implementation to properly use OpenHandsCloudWorkspace instead of directly calling the /api/conversations endpoint.

What Changed

Before:

• Cloud mode called /api/conversations directly to create a "fire and forget" conversation
• The GITHUB_TOKEN could not be passed (API rejected it)
• The cloud agent used the user's connected GitHub account

After:

• Cloud mode uses OpenHandsCloudWorkspace to provision a proper sandbox
• LLM configuration is sent to the cloud sandbox
• GITHUB_TOKEN is passed to the sandbox so the agent can post review comments
• Same capabilities as SDK mode, but running in a cloud-managed sandbox

New Requirements for Cloud Mode

Cloud mode now requires both:

• LLM_API_KEY - Sent to the cloud sandbox
• OPENHANDS_CLOUD_API_KEY - For sandbox provisioning

Documentation

See the official guide: https://docs.openhands.dev/sdk/guides/agent-server/cloud-workspace

Commits

• 10fcb495 - refactor: Use OpenHandsCloudWorkspace for cloud mode

[xingyaoww]

@OpenHands pls merge from main and resolve the conflicts

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Successfully merged main into feat/pr-review-cloud-mode and resolved all conflicts. The changes have been pushed to the remote branch.

Summary of Conflict Resolution

Three files had merge conflicts that were resolved:

1. .github/actions/pr-review/action.yml

• Merged: Cloud mode inputs (openhands-cloud-api-key, openhands-cloud-api-url) AND the new lmnr-api-key input for Laminar observability
• Merged: Both openhands-workspace package AND lmnr package in the install step
• Merged: Both cloud mode env vars AND LMNR_PROJECT_API_KEY in the run step
• Kept: github-token as required (needed for both modes)

2. examples/03_github_workflows/02_pr_review/agent_script.py

• Kept: from pydantic import SecretStr import (needed for cloud mode)
• Merged: Cloud mode logic (non-blocking run, conversation URL posting) with the improved Laminar trace handling from main (better comments explaining the trace ID usage)

3. examples/03_github_workflows/02_pr_review/README.md

• Merged: Action inputs table now includes both cloud mode inputs AND lmnr-api-key
• Kept: The "Review Evaluation (Observability)" section from main
• Updated: Input descriptions to reflect that llm-model and llm-api-key are used in both modes

The merge commit has been pushed to origin/feat/pr-review-cloud-mode.

View full conversation

[xingyaoww]

🤖 OpenHands PR Review Started

The code review is running in OpenHands Cloud.

📍 Track progress: https://app.all-hands.dev/conversations/8cff5913-4ed8-48c9-9473-e00e443de03f

The agent will post a review comment when the analysis is complete.

[xingyaoww]

✅ Cloud Mode Testing Results

I tested the cloud mode feature using the BOT_OPENHANDS_API_KEY and it works successfully!

Test Configuration

• Mode: cloud
• LLM Model: litellm_proxy/claude-sonnet-4-5-20250929
• LLM Base URL: https://llm-proxy.app.all-hands.dev
• Cloud API URL: https://app.all-hands.dev
• Review Style: roasted
• Target PR: This PR (feat: Add cloud mode for PR review workflow #1966)

Results

Step	Status
Sandbox creation	✅ 4pwQH455yV5r8w9ODkBcga
Sandbox ready	✅ Running at runtime URL
Agent trigger	✅ HTTP 200 OK
PR comment posted	✅ HTTP 201 Created

Cloud Conversation

The PR review is running here: https://app.all-hands.dev/conversations/8cff5913-4ed8-48c9-9473-e00e443de03f

Minor Observation

There were some validation warnings about an enable_truncation field (schema mismatch between local SDK models and the cloud server response). These are non-blocking - the conversation started and is running successfully. This might be worth investigating for cleaner integration, but it does not prevent the cloud mode from functioning.

pydantic_core._pydantic_core.ValidationError: 1 validation error for Event
system_prompt.enable_truncation
  Extra inputs are not permitted [type=extra_forbidden, input_value=True, input_type=bool]

Summary

The cloud mode implementation works as expected:

1. Creates a sandbox via OpenHands Cloud API
2. Sends the PR review prompt to the agent
3. Starts the agent in non-blocking mode
4. Posts a tracking comment on the PR with the conversation URL
5. Exits immediately (fast CI completion as advertised)

Great work on this feature! 🎉

[xingyaoww]

🤖 OpenHands PR Review Started

The code review is running in OpenHands Cloud.

📍 Track progress: https://app.all-hands.dev/conversations/578fcb531dd34ef3b87fe7fd55b29e46

The agent will post a review comment when the analysis is complete.

[openhands-ai]

Looks like there are a few issues preventing this PR from being merged!

• GitHub Actions are failing:
  • Review Thread Gate

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1966 at branch `feat/pr-review-cloud-mode`

Feel free to include any additional details that might help me get this PR into a better state.

_{^{You can manage your notification settings}}

[xingyaoww]

@OpenHands pls fix #1966 (review)

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

I've addressed the PR review feedback from #1966 (review). The changes have been pushed to the feat/pr-review-cloud-mode branch.

Summary of Changes

1. Fixed Documentation/Code Mismatch (Critical Issue #1)

• README.md: Updated to clarify that cloud mode requires both OPENHANDS_CLOUD_API_KEY and LLM_API_KEY (the LLM config is sent to the cloud sandbox)
• action.yml: Updated input description for llm-api-key to indicate it's required for both modes
• action.yml validation: Updated to check for LLM_API_KEY before mode-specific checks

2. Replaced Over-Engineered HTTP Client (Critical Issue #2)

• Removed the 65-line _make_http_request function
• Replaced with simple requests library calls in get_pr_diff_via_github_api() and post_github_comment()
• Added requests to the action.yml dependencies

3. Added Error Handling for Cloud Mode (Improvement #6)

• Added try/except around conversation.run() in run_cloud_mode()
• If the review fails after posting "review started", a failure comment is posted to the PR with the error message and conversation URL

4. Fixed Missing Dependency

• Added openhands-workspace package to action.yml dependencies (required for OpenHandsCloudWorkspace)

5. Updated Tests

• Removed tests for the deleted _make_http_request function
• Updated TestPostGithubComment and added TestGetPrDiffViaGithubApi to test the new requests-based implementations
• All 15 tests pass

Checklist

• Documentation/code mismatch fixed
• Over-engineered HTTP client replaced with requests
• Error handling added for cloud mode conversation failure
• Tests updated and passing
• Changes pushed to remote branch

View full conversation

[xingyaoww]

@OpenHands /codereview-roasted

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Code Review Summary for PR #1966: "feat: Add cloud mode for PR review workflow"

🔴 Taste Rating: Needs Improvement

I performed a Linus Torvalds-style "roasted" code review of this PR. Here are the key findings:

---

Critical Issues (Must Fix)

1. Design Contradiction: keep_alive=True but blocking on run() (agent_script.py, lines 219-258)

   • The code uses keep_alive=True (suggesting fire-and-forget) but then blocks on conversation.run(). This defeats the stated benefit of "faster CI completion". Pick one design.

2. Complexity: run_cloud_mode() is doing too much (~100 lines)

   • Violates single responsibility: formats prompt, creates LLM config, sets up workspace, creates agent, posts comments, handles errors with nested try/except, prints cost summary. Should be split into smaller functions.

3. Confusing API: llm-api-key marked optional but actually required (action.yml)

   • The input says required: false but validation still fails without it. Don't lie to users.

---

Improvement Opportunities

1. CLOUD_MODE_PROMPT assumes gh CLI exists - No verification that the cloud sandbox has gh installed and authenticated.

2. Inconsistent agent creation between modes - Cloud mode uses get_default_agent() while SDK mode uses Agent() with different configuration. This will cause different behavior.

3. Over-engineering: Cloud mode requires LLM_API_KEY - The whole point of cloud mode should be "just use the cloud's LLM". Requiring users to provide their own key creates unnecessary coupling.

4. Redundancy: Passing GITHUB_TOKEN to secrets - Comment says it's "automatically available" but code passes it anyway.

5. Missing integration tests - TestRunCloudMode only tests _get_required_vars_for_mode(), not the actual run_cloud_mode() function.

---

Verdict

❌ Needs rework before merging. The keep_alive=True + blocking run() contradiction is a fundamental design issue that undermines the stated benefits of cloud mode.

Key Insight

The cloud mode implementation tries to be both "fire-and-forget" and "wait for completion". Pick one design and commit to it.

---

No code changes were made - this was a review-only task as requested by /codereview-roasted.

View full conversation

[xingyaoww]

@OpenHands pls address #1966 (comment)

keep_alive=True, we should exit the script once we've hit the .run()

Confusing API: llm-api-key marked optional but actually required (action.yml)

We should fix the validation - we should only validate it on the SDK path

[xingyaoww]

@OpenHands /codereview-roasted

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Code Review Summary for PR #1966: "feat: Add cloud mode for PR review workflow"

Taste Rating: 🔴 Needs Improvement

I provided a Linus Torvalds-style "roasted" code review of this PR. No code changes were made - this was a review-only request.

---

Critical Issues Identified:

1. run_cloud_mode() complexity (Lines 179-266): 90-line function doing 9 different things - violates single responsibility principle. Should be decomposed.

2. Silent failure pattern in _get_pr_info() (Lines 401-410): Returns empty strings for missing env vars instead of failing fast. This allows the code to proceed with garbage data.

3. Awkward LLM configuration (Lines 211-216): Conditional api_key handling creates confusion about whether it's truly optional or "sometimes optional."

---

Improvement Opportunities:

• Verbose secrets dict construction could be simplified
• CLOUD_MODE_PROMPT is a fragile 36-line string with embedded bash commands
• Unused import (get_default_agent) only used in cloud mode
• Test file uses sys.path.insert() hack instead of proper module structure
• Duplicated validation logic between bash (action.yml) and Python

---

Key Insight:

The fundamental issue is that "cloud mode" and "sdk mode" have different data requirements but share the same entry point with conditional logic scattered throughout. A cleaner design would have each mode as a separate, focused module with a thin dispatcher in main().

---

Verdict: ❌ Needs rework

The core functionality is sound, but the run_cloud_mode() function needs decomposition, and the silent failure pattern in _get_pr_info() must be fixed before this is production-ready.

View full conversation

[enyst]

@OpenHands Read this PR and investigate all you need to answer me:

• what cloud account is in use in this code added by this PR?
• does the code use more than one cloud account, that is, is there a bot account and a user account, maybe who calls the bot or who comments or any other user?
• give me file/lines in a format clickable on github so I can see where any account is used.

[openhands-ai]

I only answered your questions (no code changes were made).

Answers (restated)

What cloud account is in use in this code added by this PR?

• The OpenHands Cloud account that owns the OPENHANDS_CLOUD_API_KEY. That key is used to authenticate to the OpenHands Cloud API (default https://app.all-hands.dev) and create/use the OpenHandsCloudWorkspace.

Key references:

• agent_script.py L199-L206
• agent_script.py L221-L226

Does the code use more than one cloud account (bot vs user)?

• No — only one OpenHands Cloud account is used, the one behind OPENHANDS_CLOUD_API_KEY.
• However, two GitHub identities can be involved:
  1. The identity represented by the workflow’s GITHUB_TOKEN posts the initial “tracking” PR comment.
  2. The cloud agent may use GitHub access via the token passed into conversation secrets and/or the cloud account’s GitHub integration (as described in the README).

Key references:

• post_github_comment uses GITHUB_TOKEN (agent_script.py L118-L138)
• GITHUB_TOKEN passed into cloud conversation secrets (agent_script.py L228-L239)
• README: cloud account must have GitHub access (README.md L123-L125)

Clickable links to where any account/credential is used

OpenHands Cloud credential (OPENHANDS_CLOUD_API_KEY)

• agent_script.py L199-L206
• agent_script.py L221-L226

GitHub token (GITHUB_TOKEN)

• Fetch PR diff via GitHub API (agent_script.py L65-L86)
• Post PR comment (agent_script.py L118-L138)
• Pass token into cloud conversation secrets (agent_script.py L228-L239)

Optional LLM credential (LLM_API_KEY)

• Read optional LLM_API_KEY (agent_script.py L202-L216)
• Pass optional LLM_API_KEY into secrets (agent_script.py L233-L239)

Checklist against your request

• Identified which OpenHands Cloud account is used
• Determined whether more than one cloud account is used (and clarified GitHub identity split)
• Provided actually clickable Markdown links for each credential/account usage

[xingyaoww]

@OpenHands please merge from main and resolve all conflict, KEEP existing logic from the main branch. We should refactor the agent script to have something that calls run_agent_review and it can in a separate file with either SDK implementation OR cloud implementation using OpenHandsCloudWorkspace

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Summary

I've successfully merged from main into feat/pr-review-cloud-mode and refactored the PR review agent script as requested. The changes have been pushed to the remote branch.

What was done:

1. Merged from main - Brought in all the latest changes from the main branch including:

   • GraphQL-based review context fetching (reviews and threads)
   • format_prompt function from prompt.py
   • Laminar trace support
   • Various SDK updates

2. Refactored the agent script into a modular structure:

   • agent_script.py - Main entry point that:

     • Keeps all the existing logic from main (GraphQL review context, diff fetching, etc.)
     • Determines the mode (sdk or cloud) from the MODE environment variable
     • Calls run_agent_review() from the appropriate mode module

   • sdk_mode.py - SDK implementation that:

     • Runs the review locally using the OpenHands SDK
     • Requires LLM_API_KEY for local LLM execution
     • Includes Laminar trace support and cost summary

   • cloud_mode.py - Cloud implementation that:

     • Uses OpenHandsCloudWorkspace to run reviews in OpenHands Cloud
     • Posts a tracking comment with the conversation URL
     • Runs non-blocking (exits immediately after starting)
     • LLM_API_KEY is optional (uses cloud's configured LLM)

3. Updated tests - Modified test_pr_review_agent.py to reflect the new module structure

Files changed:

• examples/03_github_workflows/02_pr_review/agent_script.py - Refactored main script
• examples/03_github_workflows/02_pr_review/sdk_mode.py - New SDK mode implementation
• examples/03_github_workflows/02_pr_review/cloud_mode.py - New cloud mode implementation
• tests/github_workflows/test_pr_review_agent.py - Updated tests

The PR is now updated at: #1966

View full conversation

[all-hands-bot]

[Automatic Post]: It has been a while since there was any activity on this PR. @xingyaoww, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

[xingyaoww]

Too much conflict, gonna restart