Web and social reading tools for coding agents
Agent Reach is a Python CLI that sets up a coding agent (Claude Code, Cursor, OpenClaw) to read and search the web and social platforms: YouTube, GitHub, X, Reddit, LinkedIn, Instagram and several Chinese sites. Agent Reach does no reading of its own. The project calls itself a "capability layer": for each platform it picks an open source tool, installs it and checks that it responds, then the agent calls that tool directly (yt-dlp for YouTube, gh for GitHub, Jina Reader for web pages). A SKILL.md file dropped into the agent's skills folder tells it which command to run.
The comparison of open source web scraping tools sets out what each one assumes in terms of browser, proxies and accounts.
Ranked fallbacks and an installer that asks before acting
- Fallback tools for each platform: every channel ranks its tools in order of preference. In June 2026 Bilibili was rejecting yt-dlp with a 412 error, and the channel switched to bili-cli.
agent-reach doctor: the health check actually runs each tool instead of looking for the binary, flags tools that are installed but broken, and prints the command that fixes them.- An installer that asks first: without
--system,agent-reach installinspects the machine and changes nothing.--dry-runlists what it would do. - Channels that need no account: web pages, YouTube subtitles and search, public GitHub repos, RSS feeds, Exa search, the V2EX forum and Bilibili search.
- Chinese platforms: Bilibili, Xiaohongshu, V2EX, Xueqiu stock quotes, Boss Zhipin job listings and Xiaoyuzhou podcasts, transcribed with Whisper. The main README is in Chinese.
Cookies from a logged-in account, ideally a spare one
Most social networks require a logged-in account. Searching X takes the cookies of an account, exported by hand with the Cookie-Editor extension. Reddit has no anonymous route left: logged-out endpoints are blocked and the official API needs manual approval. Facebook, Instagram and Xiaohongshu reuse the user's Chrome session, while Xueqiu, Boss Zhipin and full LinkedIn profiles also need a login. Transcription needs a Groq API key, which is free.
The project recommends a secondary account. The README warns that platforms can detect these scripted calls and restrict or ban the account, and that a cookie amounts to full account access. It says nothing about the sites' terms of service. Credentials stay in ~/.agent-reach/config.yaml with 600 permissions.
Several channels assume a desktop with Chrome. OpenCLI, which handles Reddit, Facebook, Instagram and Xiaohongshu, installs through npm plus a Chrome extension the user approves by hand, and the installer skips it on servers. Boss Zhipin does not run without a desktop. Node.js, gh and mcporter are on the prerequisite list. The PyPI package of the same name is a different project: installation goes through the GitHub archive.
Reading and search go through two hosted services. The web channel sends every URL to r.jina.ai, Jina Reader's API. When a Cloudflare challenge page comes back, it stops and points to a site-specific tool or a browser. Camoufox targets that case: a Firefox fork that draws a fresh device fingerprint on every launch and hides the automation from page scripts. Web search goes through Exa's remote MCP server. OpenSERP runs on the user's own machine instead and queries Google, Bing or DuckDuckGo with no API key.
Beta, with a catalog that keeps changing. pyproject.toml lists the project as "4 - Beta". The latest release, v1.5.0, came out on June 11, 2026, the seventh since February. Installation pulls the main branch, though, which has moved on: the current Boss Zhipin channel was merged there on September 15, with no release since. Version 1.4.2 dropped Douyin, Weibo and WeChat official accounts until a reliable upstream tool exists for them.