Skip to content

Add server mode with live reload (rdoc --server)#1620

Open
st0012 wants to merge 8 commits intomasterfrom
server-mode-retry
Open

Add server mode with live reload (rdoc --server)#1620
st0012 wants to merge 8 commits intomasterfrom
server-mode-retry

Conversation

@st0012
Copy link
Member

@st0012 st0012 commented Feb 21, 2026

A better attempt of #1151

Implement rdoc --server[=PORT] for previewing documentation with automatic browser refresh on source file changes. The server parses all sources on startup, serves pages from memory via the Aliki generator, and watches for file modifications, additions, and deletions — re-parsing only what changed.

rdoc-server-demo.mp4

Changes

  • New RDoc::Server (lib/rdoc/server.rb) — minimal HTTP server using Ruby's built-in TCPServer (no WEBrick or external dependencies)
    • Thread-per-connection with Connection: close
    • Persistent Aliki generator instance rendering pages to strings
    • In-memory page cache and search index cache with full invalidation on changes
    • Live reload via inline JS polling /__status endpoint every 1 second
    • Background file watcher polling source file mtimes every 1 second
    • Incremental re-parse: only changed files are re-parsed, old data removed first
    • Detection of new and deleted source files
  • --server[=PORT] CLI option (default port 4000) and rdoc:server Rake task
  • RDoc::Store#remove_file — removes a file's entries from the store hashes
  • RDoc::Store#clear_file_contributions — surgically removes a file's methods, constants, comments, includes, extends, and aliases from its classes/modules, preserving classes that span multiple files
  • RDoc::RDoc#relative_path_for — extracted path normalization (against options.root and options.page_dir) shared by parse_file and the server
  • Darkfish#refresh_store_data — extracted for reuse by the server after re-parsing
  • RDoc::ServletRDoc::RI::Servlet — moved to clarify RI-specific usage

Security & robustness

  • Binds to 127.0.0.1 only (localhost)
  • Path traversal protection in asset serving (File.expand_path containment check)
  • Proper HTTP error responses: 400, 404, 405, 500
  • 5-second IO.select read timeout on client sockets
  • Mutex protects all store mutations, generator refresh, and cache invalidation atomically
  • Individual parse_file errors rescued so one failure doesn't block remaining files
  • Watcher thread uses @running flag with clean shutdown via Thread#join

Known limitations

  • Full cache invalidation: any file change clears all cached pages. Rendering is fast (~ms per page); parsing is the expensive part and is done incrementally.
  • Template/CSS changes: require server restart (only source files are watched).

@matzbot
Copy link
Collaborator

matzbot commented Feb 21, 2026

🚀 Preview deployment available at: https://21ad6ef5.rdoc-6cd.pages.dev (commit: 28b884f)

Adds a built-in HTTP server for previewing documentation while editing
source files. Parses all sources on startup, watches for file changes,
re-parses only changed files, and auto-refreshes the browser.

Server implementation (lib/rdoc/server.rb):
- Uses Ruby's built-in TCPServer (no WEBrick or external dependencies)
- Persistent Aliki generator instance rendering to strings
- Thread-per-connection with Connection: close (no keep-alive)
- Background watcher thread polls file mtimes every 1 second
- Live reload via inline JS polling /__status endpoint
- New --server[=PORT] option (default 4000) and rdoc:server Rake task
- Moved RDoc::Servlet to RDoc::RI::Servlet (server mode uses new class)

Security:
- Binds to 127.0.0.1 only (localhost)
- Path traversal protection in asset serving via expand_path containment
- Proper HTTP error responses (400, 404, 405, 500)
- 5-second read timeout on client sockets

Concurrency:
- Mutex protects all store mutations, generator refresh, and cache
  invalidation as a single atomic operation
- Thread-safe last_change_time reads for the status endpoint

Correctness:
- Clears file contributions (methods, constants, comments, etc.) before
  re-parsing to prevent duplication, without removing shared namespaces
- Individual parse_file errors caught so one failure doesn't block others
- Store#remove_file recursively cleans nested classes/modules and C vars
- Watcher thread uses @running flag with clean shutdown via join
@st0012 st0012 marked this pull request as ready for review February 21, 2026 17:58
Co-authored-by: Sutou Kouhei <kou@clear-code.com>
- Embed last_change_time into the live-reload script at render time
  so the browser's initial timestamp matches the page content. This
  fixes a race where a change between page generation and the first
  poll would be silently skipped.

- Call clear_file_contributions for removed files (not just changed
  files) and remove classes/modules from the store when no files
  contribute to them anymore. This correctly handles reopened classes
  across multiple files and improves file deletion behavior.
Move `relative_path_for` from a private method on RDoc::Server to a
public method on RDoc::RDoc, eliminating the duplication with the
inline logic in `parse_file`.

Move `clear_file_contributions` from RDoc::Server to RDoc::Store
where it naturally belongs — it operates entirely on store internals
(files_hash, classes_hash, modules_hash).

Add tests for Store#clear_file_contributions covering single-file
removal, multi-file preservation, per-file cleanup of methods/
constants/includes, and no-op for nonexistent files.
@st0012 st0012 requested review from kou and tompng February 28, 2026 15:06
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

# server.mount '/rdoc', RDoc::RI::Servlet, '/rdoc'

class RDoc::Servlet < WEBrick::HTTPServlet::AbstractServlet
class RDoc::RI::Servlet < WEBrick::HTTPServlet::AbstractServlet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion whether we re-implement our HTTP server or we use WEBrick but do we want to keep depending on WEBrick?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can let it go and potentially merge the server implementation between RDoc and RI. But it'll be outside of the scope of the PR.

st0012 added 4 commits March 1, 2026 14:48
- Use exit instead of return in document method for consistency
- Replace path.sub regex with delete_prefix in server.rb
- Use to_json for safe JavaScript value embedding
The search index was double-cached in both @search_index_cache and
@page_cache. Remove the dedicated cache and inline the one-liner into
generate_page, letting @page_cache handle all caching.

Also cache @template_dir in initialize instead of recomputing
File.expand_path on every asset request.
Convert ClassModule#comment_location from an Array of [comment, location]
pairs to a Hash of { location => comment }. Ruby hashes preserve insertion
order, and replacing an existing key preserves its position, which naturally
fixes the comment reordering bug during server re-parse without needing
empty placeholder workarounds.

Key changes:
- add_comment simplifies to a single hash assignment
- clear_file_contributions gains keep_position: keyword for server re-parse
- C parser's delete_if special case is no longer needed (hash replaces)
- Same-file duplicate comments are naturally deduplicated
- Marshal format is unchanged (serialized via parse() as before)
@name = name
@superclass = superclass
@comment_location = [] # Array of [comment, location] pairs
@comment_location = {} # Hash of { location => comment }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same location comment will be removed with this change.

Example input:

# comment1
class A; end
# comment2
class A; end

Output:

<section class="description">
<p>comment1</p> <!-- this line disappears -->
<p>comment2</p>
</section>

I don't think these kind of reopening with comment is normally used, but the original deduplicating logic is clearly only for C code.

if location.parser == RDoc::Parser::C
  @comment_location.delete_if { |(_, l)| l == location }
end

Since reopening class with a comment is both included in the document in some case(different file), I think it's natural to accept the same thing written in a single file too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants