API Documentation
Page Endpoint
https://statsify.herokuapp.com/pageOptions:
url (required) - the desired url.
Example Call:
https://statsify.herokuapp.com/page?url=http://example.comResponse:
Website Endpoint
https://statsify.herokuapp.com/websiteOptions:
url (required) - the desired url.
depth (optional) - the maximum
depth of links to recursively crawl through before stopping.
- default = 1
- 0 ≤
depth≤ 8
Example Call:
https://statsify.herokuapp.com/website?url=http://example.com&depth=1Response:
Note 1: The "page" key contains an object with the same structure
as returned by the page endpoint.
Note 2: "freq" refers to how many times a page is linked to across the whole website.
Errors
Example Response:
Common Errors:
- Status Code 200 - Status is okay, but the attempt to request the given url failed.
- Status Code 200 - Status is okay, but robots are restricted from accessing this url.
- Status Code 404 - Endpoint not found.
- Status Code 429 - Rate limit reached (page: ~1000/day, website: ~200/day).
- Status Code 500 - An error occurred on the server.
Other Details
- Currently, only HTML pages are included in the data returned by the API.
- The API stops scraping new sites after reaching 100 unique urls or after 25 seconds have passed in a single call to prevent reaching Heroku's 30 second timeout limit. The 100-url limit includes non-HTML pages, as each page takes time to request.
- After the 100-url or 25-second limit is reached, all other urls are grouped together under the url * with the following JSON representation (does not affect a website's average word count):