x-tweet-by-handle

X — Tweets by Handle

> X handle + timeline mode → normalized list of that user's tweets (or replies / media-only) with author, engagement, media, cursor.

Language

All process output to user (progress updates, process notifications) follows the user's language.

Objective

Collect tweets from one or more X user profile timelines, selectable between the default tweets tab, the tweets-and-replies tab, and the media-only tab, returning structured per-tweet data and pagination cursors.

Prerequisites

• Active X session in the browser (left sidebar shows logged-in avatar / @handle).
• Network capture is enabled in the browser-act session.

Pre-execution Checks

1. Tool Readiness

If browser-act has been confirmed available in the current session → skip this step.

Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.

2. Login Verification

If login status for X has been confirmed in the current session → skip this step.

Otherwise: open https://x.com and observe the left sidebar:

• User avatar or @handle visible → logged in, continue
• "Sign in" / "Log in" prompt visible → not logged in, inform the user and assist the login flow

User refuses or cannot log in → terminate execution.

Capability Components

> This Skill's operational boundary = what the user can manually do in their browser. It only reads tweet data already shown to the user, never bypassing authentication. The browser's own JS signs the GraphQL request; the Skill triggers it via URL navigation and reads the response from network traffic. Python scripts under scripts/ only build URLs and parse responses — they do not call X directly. Run them through the bash tool.

Network Capture: profile timeline (tweets / replies / media)

Step 1 — build the profile URL for the desired mode:

URL=$(python scripts/build-profile-url.py '{handle}' --mode {mode})

Parameters:

• handle (positional): X handle without @, case-insensitive.
• --mode: one of tweets (default — triggers UserTweets), replies (triggers UserTweetsAndReplies), media (triggers UserMedia).

Step 2 — navigate and capture the first page:

1. network requests --clear
2. navigate "$URL"
3. wait stable --timeout 25000 (timeout is normal on X; proceed even if it fires)
4. Determine the endpoint name to filter by — UserTweets, UserTweetsAndReplies, or UserMedia matching the chosen mode:

• network requests --type xhr,fetch --filter UserTweets (when --mode tweets)
• network requests --type xhr,fetch --filter UserTweetsAndReplies (when --mode replies)
• network requests --type xhr,fetch --filter UserMedia (when --mode media)

1. Take the latest matching entry's request_id.
2. network request <request_id> → save full output to a file (e.g. tmp/x-profile-page-1.txt).
3. Parse with the matching --source flag:

• python scripts/parse-tweets.py --json-file tmp/x-profile-page-1.txt --source user_tweets (for tweets)
• python scripts/parse-tweets.py --json-file tmp/x-profile-page-1.txt --source user_replies (for replies)
• python scripts/parse-tweets.py --json-file tmp/x-profile-page-1.txt --source user_media (for media)

Endpoint characteristic: URL contains /i/api/graphql/<hash>/UserTweets (or UserTweetsAndReplies / UserMedia). The query hash rotates; always filter by name.

Step 3 — paginate via scroll:

1. network requests --clear
2. scroll down --amount 5000
3. wait stable --timeout 10000
4. network requests --type xhr,fetch --filter <EndpointName> → take the newest entry's request_id
5. network request <request_id> → save to tmp/x-profile-page-N.txt
6. python scripts/parse-tweets.py --json-file tmp/x-profile-page-N.txt --source <source>

Repeat Step 3 until any termination condition is met:

• Accumulated unique tweet count reaches the user's target.
• count == 0 on the current page.
• cursor_bottom is unchanged across two consecutive pages.

Error handling: if no matching request appears after a scroll, wait 3 s and retry once. If the page redirects to a "This account doesn't exist" or "Account suspended" view (visible in state output), terminate with an explanatory result. Pinned tweets appear at the top of the first page — they are normal tweets and are included in the output; de-duplicate by id when merging if a pinned tweet would also appear later in chronological order.

Output example:

{
  "tweets": [
    {
      "type": "tweet",
      "id": "2068333045510291908",
      "url": "https://x.com/NASA/status/2068333045510291908",
      "twitter_url": "https://twitter.com/NASA/status/2068333045510291908",
      "text": "The official FIFA World Cup ball went to space! ...",
      "created_at": "Thu Jun 20 18:30:11 +0000 2026",
      "lang": "en",
      "source": "Twitter Web App",
      "retweet_count": 4586,
      "reply_count": 1812,
      "like_count": 24499,
      "quote_count": 240,
      "bookmark_count": 1730,
      "view_count": 2098235,
      "is_reply": false,
      "is_retweet": false,
      "is_quote": false,
      "quote_id": null,
      "quote_url": null,
      "in_reply_to_id": null,
      "in_reply_to_user": null,
      "in_reply_to_user_id": null,
      "conversation_id": "2068333045510291908",
      "hashtags": ["FIFAWorldCup"],
      "mentions": [],
      "urls": [],
      "media": [
        {"type": "photo", "url": "https://pbs.twimg.com/media/abcd.jpg", "expanded_url": "https://x.com/NASA/status/2068333045510291908/photo/1", "alt_text": null}
      ],
      "card": null,
      "place": null,
      "author": {
        "id": "11348282",
        "user_name": "NASA",
        "name": "NASA",
        "url": "https://x.com/NASA",
        "is_verified": false,
        "is_blue_verified": true,
        "verified_type": "Government",
        "profile_picture": "https://pbs.twimg.com/profile_images/.../photo.jpg",
        "description": "Explore the universe ...",
        "location": "Pale Blue Dot",
        "followers": 92137231,
        "following": 305,
        "created_at": "Wed Dec 19 20:20:32 +0000 2007"
      }
    }
  ],
  "count": 20,
  "cursor_top": "DAAHCgABHLo27Q8__-s...",
  "cursor_bottom": "DAAHCgABHLo27Q8__-s..."
}

Pagination

Network Capture Pagination: triggered by scroll down. X's page JS inserts the previous response's cursor_bottom into the next request's variables.cursor. Termination: count == 0, cursor_bottom does not advance across two consecutive pages, or user target reached.

Success Criteria

count >= 1 on the first page (unless the account has zero tweets in the chosen mode) AND every tweet has non-null id, text, created_at, author.user_name, like_count, retweet_count, reply_count AND each subsequent page's cursor_bottom differs from the previous page's until termination.

Known Limitations

• Protected (locked) accounts return zero tweets unless the logged-in session follows that account.
• Suspended or deleted accounts return an error page; the Skill will detect via empty count and a redirect to /account_suspended or /account/access and terminate the loop.
• view_count is null for very new or low-traffic tweets where X has not yet emitted views.count.
• --mode media returns tweets that contain native media only; pure-text or quote-only tweets are skipped by X server-side.
• Sustained polling triggers per-session throttling; stay under ~150 timeline calls per 15-minute window per session (x-rate-limit-remaining shows the live budget).
• The page may never reach network-idle; wait stable will frequently time out — proceed to read network anyway.

Execution Efficiency

• Batch orchestration: write a bash script that iterates handles serially in one session; for parallelism, fan out across multiple stealth browsers each with its own login.
• Test before batch execution: run one handle end-to-end (page 1 + page 2) first.
• Reduce redundant pre-operations: keep the same session for many sequential handles — navigate reuses the SPA.
• Error resumption: persist cursor_bottom + accumulated tweet IDs to disk after every page; a crash mid-batch resumes from the last good cursor.
• De-duplicate by id: pinned tweets reappear when chronological order catches up; merge by id.

Experience Notes

Path: {working-directory}/browser-act-skill-forge-memories/x-tweet-scraper-x-tweet-by-handle.memory.md (working directory is determined by the Agent running the Skill, typically the project root or current working directory)

Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.

After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:

{YYYY-MM-DD}: {what happened} → {conclusion}

Normal execution does not write to the file. Do not record what handle was scraped or how many tweets were returned — those are task outputs, not experience.