flyscrape for Mac

Flyscrape is a command-line web scraping tool designed for those without advanced programming skills, enabling precise extraction of website data.

Installation · Documentation · Releases

Demo

Features Standalone: Flyscrape comes as a single binary executable. jQuery-like: Extract data from HTML pages with a familiar API. Scriptable: Use JavaScript to write your data extraction logic. System Cookies: Give Flyscrape access to your browsers cookie store. Browser Mode: Render JavaScript heavy pages using a headless Browser. Nested Scraping: Extract data from linked pages within a single scrape.

Overview Example Installation Recommended Homebrew Pre-compiled binary Compile from source Usage Configuration Query API Flyscrape API Document Parsing File Downloads Issues and suggestions

Example

This example scrapes the first few pages form Hacker News, specifically the New, Show and Ask sections.

Check out the examples folder for more detailed examples.

Installation

Recommended

The easiest way to install flyscrape is via its install script.

Homebrew

For macOS users flyscrape is also available via homebrew:

Pre-compiled binary

flyscrape is available for MacOS, Linux and Windows as a downloadable binary from the releases page.

Compile from source

To compile flyscrape from source, follow these steps: Install Go: Make sure you have Go installed on your system. If not, you can download it from https://go.dev/. Install flyscrape: Open a terminal and run the following command:

Usage

Configuration

Below is an example scraping script that showcases the capabilities of flyscrape. For a full documentation of all configuration options, visit the documentation page.

Query API

Flyscrape API

Document Parsing

File Downloads

Issues and Suggestions

If you encounter any issues or have suggestions for improvement, please submit an issue.