Over 1,100 stories hit Hacker News in the 24 hours covered by the tutorial this article is based on. A Hacker News scraper built with ChatGPT and ScrapeAtlas filtered that volume into a handful of highlighted tools in about three and a half minutes.
What a Hacker News scraper built with ChatGPT actually does
A Hacker News scraper built inside ChatGPT is a connector setup: ChatGPT queries a third-party scraping service, ScrapeAtlas, which fetches structured data from Hacker News and returns it so the model can summarize, filter, and rank it. The model itself never scrapes the site; it only interprets the data the connector retrieves.
Hacker News is the link-aggregation and discussion forum run by Y Combinator, the startup accelerator. It has kept roughly the same plain design since its launch in 2007, and it remains a dense source of early signals on AI tools, developer tooling, and startups. The value of pairing it with a language model is synthesis: in the video this article is based on, a single daily run processed over 1,100 new stories and produced a short executive summary of the ones worth a closer look.
One correction to the video's framing: it credits the result to a model it calls "GPT 6 Astra." OpenAI has not published a model under that name on its official site as of October 2026, so this article describes the workflow itself and attributes the model claim to the video's creator rather than to OpenAI.
Why ChatGPT cannot scrape Hacker News directly
ChatGPT does not scrape Hacker News directly because automated crawling of the site conflicts with its robots.txt instructions and general terms of use. The video creator gives the correct reason: many platforms restrict which crawlers may access them, and valuable discussion data is exactly the kind these restrictions protect.
That is why the workflow needs an external scraping service acting as a middle layer. The service performs the data collection under its own compliance practices, then hands ChatGPT clean, structured records. ChatGPT's role stays where it is strongest: reading large volumes of stories and comments and turning them into a short, relevant report.
Be aware of the boundary here. Delegating the scraping to a vendor does not change the underlying question of whether a given use of the data is permitted; it only changes who runs the crawler. For personal monitoring and research this is routine practice, but commercial reuse of scraped content deserves its own review.
How to connect ScrapeAtlas to ChatGPT via MCP
You connect ScrapeAtlas to ChatGPT by enabling developer mode, adding ScrapeAtlas as a plugin through its MCP link, and permitting all of its actions. ScrapeAtlas is a scraping service covering social platforms, marketplaces, ad libraries, and Hacker News, and it exposes its data through MCP, the Model Context Protocol that lets assistants call external tools. The video reports 100 free credits on signup, which it says is enough to get the scraper running.
The setup takes about five steps:
- In ChatGPT, open Settings, then Security and login, and toggle Developer mode on. This is what unlocks custom plugin connections.
2. Log in at ScrapeAtlas, go to Integrations, and copy the MCP link.
3. In ChatGPT's Plugins panel, click the plus sign, name the connection, paste the MCP link, and click Create.
4. Open the plugin's three-dot menu, choose Manage, and expand permissions from low-risk actions to all actions. Scraping is read-only, so the video considers this safe, but you are approving whatever actions the vendor ships later, so review before you widen the scope.
5. Start a new chat and confirm the ScrapeAtlas logo appears in the plugin list.
The example in the video runs inside ChatGPT's Work mode with a paid subscription; the creator says the regular chat window did not expose the model he used. That detail may change as OpenAI revises its plans, so check current plan limits rather than assuming the video's configuration still applies.
Which Hacker News endpoints and data points you can pull
The ScrapeAtlas Hacker News integration exposes the same data that is publicly listed on the site: story text, URL, story ID, upvotes, and comments, organized into endpoints you can inspect in the vendor's documentation. Everything ChatGPT can reason about comes from those records.
The practical building blocks look like this:
| Endpoint | Data returned | Typical use |
|---|---|---|
| Stories | Title, text, URL, story ID, upvotes | Daily digest of new posts |
| Comments | Comment text, author, story reference | Sentiment and reactions per story |
| Ranked listing | Stories ordered by score | Trend detection across a time window |
Combining endpoints is where the workflow gets useful. A stories-only monitor surfaces new tools fast but misses reception; adding comments lets the model report what practitioners actually think. The video creator deliberately kept his first monitor product-focused and noted he could widen it to comment sentiment later by adjusting the prompt.
Building the daily Hacker News monitor inside ChatGPT
You build the daily monitor by prompting ChatGPT in Work mode to explore the ScrapeAtlas endpoints and create a recurring job. In the video, the whole construction took one prompt and roughly three and a half minutes of processing before the monitor appeared with its first report.
The prompt that worked followed a simple pattern: name the connector, state the goal, and give the model a relevance filter. The creator's version read, in essence: build a daily monitor over new Hacker News stories via ScrapeAtlas, synthesize the most important ones, and prioritize automation content and demoable AI tools because he runs a YouTube channel on those topics. The specificity mattered. Without the channel context, the model would have summarized everything and highlighted nothing.
Once created, the monitor is adjustable through its controls: you can pause it, run it on demand, edit the prompt, or change the cadence. The first run in the video processed more than 1,100 stories from the previous 24 hours and returned a handful of highlighted products with links and short evaluations, which is the real payoff: an LLM sifting volume into a personally filtered reading list.
Other use cases the same setup supports
The same connector supports any task that reduces to reading Hacker News at scale. The video names several, and each maps to a different prompt and endpoint combination:
- Startup research: watch for new launches in a sector and pull founder commentary from comment threads.
- Sentiment analysis: pair story endpoints with comment endpoints to gauge how a tool or announcement landed.
- Competitor monitoring: filter for mentions of specific product names or domains.
- Trend tracking: rank a time window by upvotes to see what gained traction, not just what appeared.
- Keyword monitoring: restrict the digest to stories matching chosen terms.
Two limits are worth stating plainly. Coverage is Hacker News only in this example; ScrapeAtlas lists other platforms, but each is its own endpoint set. And a monitor is only as good as its relevance filter, so expect to iterate on the prompt a few times before the daily report matches what you would have chosen yourself.
FAQ
- Do I need developer mode enabled to use ScrapeAtlas in ChatGPT? Yes. Developer mode, found under Settings and then Security and login, is what allows custom plugin connections via an MCP link. Without it, the plus-sign plugin flow shown in the tutorial is not available.
- Is scraping Hacker News legal? The video does not address legality, and this workflow delegates crawling to ScrapeAtlas under its own practices. For personal monitoring this is common, but check Hacker News's terms and your jurisdiction before any commercial reuse of the data.
- How much does the setup cost? The video reports 100 free ScrapeAtlas credits at signup, enough to build and test the scraper, plus a paid ChatGPT subscription for the model used in Work mode. Confirm current pricing on both sites, since plans change.
- Can the monitor also analyze comments and sentiment? Yes, if you extend the prompt to include the comments endpoint. The creator kept his first version product-only and treated comment sentiment as a later adjustment.
- Who is this workflow a good fit for? Anyone who needs a filtered view of a high-volume tech discussion feed: content creators hunting tools, analysts tracking launches, or founders watching a niche. If you read Hacker News manually for an hour a day, a monitor is the alternative.
- Is the same pattern usable with other assistants? The connector side is built on MCP, an open protocol, so other MCP-compatible assistants can in principle call the same ScrapeAtlas link. The video only demonstrates the ChatGPT path, and plugin or developer-mode naming differs between assistants.
- Does the model name in the video matter for the result? The creator attributes his run to a model he calls GPT 6 Astra; that name does not appear in OpenAI's official model listings as of October 2026. The workflow depends on a capable model plus the connector, not on that specific label.
Turn your own videos into reading material
This article exists because a seven-minute tutorial contained a complete, reproducible workflow that most viewers would scroll past once and forget. That is the same problem many creators have: their best explanations live in video, where they are hard to search, cite, or skim.
If you have workflow demos, interviews, or technical walkthroughs sitting on your YouTube channel, you can turn them into structured written articles. Visit Skala Blog, paste a YouTube URL, and the video is transcribed and reshaped into a publishable article. Related projects worth a look include the Crazystack TypeScript resources for developers building on similar automation stacks.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits