Submit a query and get back what Yahoo showed, as a table - the rank, the title, the link and the snippet, one row per result. Five columns, and in the 360 rows behind this page every one of them was filled on every row.
Across all 360 rows we hold, query, position, title, url and description were populated on every single row. Not one empty cell in the whole export. That is unusual enough to be worth stating - most sources leave you writing null checks.
In all five runs position ran 1, 2, 3 … N with no gaps and no repeats. So you can sort on it directly, and a missing number means a row was lost in your pipeline rather than in the scrape.
This is the one that catches people. Within a single run the url column repeats - 20 duplicate rows in one 74-row export, and 4, 9, 11 and 16 in the others. They are real rows at real positions, not an export bug. Dedupe on url before you count anything.
We ran python web scraping twice, 24 hours apart. 66 distinct URLs the first day, 62 the second, and only 42 in both - roughly a third turned over. Of the 42 that survived, 22 sat at a different position. Treat a single run as a snapshot, not as the ranking.
Descriptions ran from 6 to 417 characters, median 194, and 79 of the 360 ended in an ellipsis - Yahoo cut them off. Plan for a wide text column, and do not treat a trailing "…" as a data problem.
Nine of the 360 titles began with a spaced-out breadcrumb domain, like www. python .org › downloads Download Python. It is a minority, but if you match on titles it will surprise you. Strip the leading domain fragment or match on url instead.
Read from the header of a real export of this service, in sheet order. Every description below is grounded in the 360 rows we measured.
www. python .org › downloads before the real title.https and 2 were http, across 168 distinct hosts. This column repeats within a run - dedupe on it.Five columns per row · CSV, JSON or Excel
Measured across 5 real runs and 360 rows: python (74 rows), snake (69), titan (74) and python web scraping twice (70 and 73). Row count per query varied between 69 and 74, so there is no fixed page size to rely on - read the rows you get rather than assuming a round number.
Run the same queries on a schedule and keep query, url, position and the run date. Our own two runs a day apart moved 22 of 42 shared URLs, so a single reading tells you very little and a series tells you a lot.
Group by host and count. Our 360 rows spanned 168 distinct hosts, which is the shape of the question - who keeps appearing across the queries you care about, and at what depth.
Run the same query list through this and through the Google Search Scraper, then diff the URL sets. Where the two disagree is usually where an audit finds something.
A query like a category plus a city returns provider pages by the dozen. Take the url column into the Email & Contact Scraper and the SERP becomes a contactable list.
One-time, on signup. No card, and nothing to cancel afterwards.
Billed on rows actually returned. At the 69 to 74 rows a query returned in our runs, 500 free rows is roughly seven queries before anything is charged.
Credits do not expire on a monthly cycle. Run a batch now and nothing again until you need it.
LIVESCRAPER10 at checkout.The same job on Google. Run both over one query list and diff the URL sets to see where the engines disagree.
News results rather than web results, when what you need is coverage over time.
Interest over time for the terms you are ranking against - the demand side of the same question.
Feed the url column in and get contact details out, turning a results page into a list you can act on.
A results page is a public page, and the five columns here describe web pages rather than people. The caveats are about volume and reuse.
Reading a search results page is what every visitor does, and a rank, a title, a link and a snippet are facts about publicly reachable web pages. Nothing in the export identifies a person: there is no name, no email, no phone number and no account, so the data-protection questions that shape our people-facing services do not arise here.
Two things are worth stating plainly. First, snippets and titles are written by the sites being indexed, not by us, and they are those sites' copyrighted text. Using them internally for rank tracking, competitive analysis and audits is ordinary practice; republishing them as your own content is a different question with a different answer. Second, this is a snapshot and not a ranking: our own two runs of the same query a day apart shared only 42 of roughly 64 URLs, so any conclusion drawn from a single export is weaker than it looks. Run a series.
Our own terms are the same as on every other service here. Publicly available pages only, nothing behind a login, no third-party trackers on the data layer, and exports auto-delete after 30 days. Your first 500 rows are free and need no credit card.
query, position, title, url and description. One row per result. Across the 360 rows behind this page every one of those five was filled on every row - there was not a single empty cell.url before you count hosts or measure share of results.python web scraping twice, 24 hours apart: 66 distinct URLs the first day, 62 the second, 42 present in both. Of those 42, 22 had moved position. Even the top ten shared 9 of 10 URLs but not in the same order. Treat one run as a snapshot and track a series if you need a trend.url, which does not have the problem.Five columns per result, filled on every row. Your first 500 rows are free, then $0.002 each.
The Yahoo Search Scraper takes a query and returns the results page as rows: query, position, title, url and description, one row per result. Five runs and 360 rows sit behind this page, and every one of those five columns was populated on every one of those rows - no empty cells anywhere in the export. position comes back as a number running contiguously from 1, so it sorts directly and a gap in your data means a row was lost downstream rather than during the scrape.
Two properties of the data matter more than the schema. The first is duplication: within a single run the url column repeats, 20 times in one 74-row export and 4, 9, 11 and 16 times in the others. These are genuine rows at genuine positions, so any count of hosts, domains or share of results has to dedupe first. The second is volatility. We ran the query python web scraping on two consecutive days: 66 distinct URLs the first day, 62 the second, and only 42 appeared in both. Of those 42 survivors, 22 had changed position, and even the top ten shared nine of ten URLs in a different order. A single export is a snapshot of one moment, and a rank-tracking programme built on one reading per quarter will report noise as movement.
Row counts are not fixed either - our five runs returned 74, 70, 73, 69 and 74 rows for one query each. Snippets ran from 6 to 417 characters with a median of 194, and 79 of the 360 ended in an ellipsis where Yahoo truncated them, so the description column wants a wide text type rather than a short varchar. Titles are usually the page title, but on 9 of the 360 rows they opened with a spaced display domain such as www. python .org › downloads; matching on url avoids that entirely. Across all 360 rows there were 168 distinct hosts, 358 https links and 2 http.
The common uses follow from the shape. Rank tracking works if you run a series rather than a single export. Competitive visibility is a group-by on host. Coverage auditing means running the same query list through this and through the Google Search Scraper and diffing the URL sets, because where two engines disagree is usually where something interesting sits. And a results page for a category-plus-city query is a lead list once the url column goes through the Email & Contact Scraper. On the legal side, a rank, a link and a snippet are facts about public web pages and none of the five columns describes a person - though the snippets are the indexed sites' own copyrighted text, which makes internal analysis ordinary and republication a separate question. Publicly available pages only, no third-party trackers on the data layer, and exports auto-delete after 30 days. Your first 500 rows are free and need no credit card.