found 2,289 packages in 23ms

crawler17.5kMIT2.0.2

Crawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.

bda-research3 months agocrawler, javascript, spider, scraper

A library for efficiently walking a directory recursively

nodelib9 months agocrawler, NodeLib, fs, FileSystem

fdir131mMIT6.5.0

The fastest directory crawler & globbing alternative to glob, fast-glob, & tiny-glob. Crawls 1m files in < 1s

thecodrr21 days agocrawler, util, os, sys

isbot7mUnlicense5.1.30

🤖/👨‍🦰 Recognise bots/crawlers/spiders using the user agent string.

omrilotan18 days agocrawlers, bot, spiders, googlebot

pdf-parse5mMIT1.1.1

Pure javascript cross-platform module to extract text from PDFs.

autokentover 3 years agopdf-crawler, pdf-parse, xpdf, pdf.js

Packages

Lightweight debugging utility for Node.js and the browser

Lodash modular utilities.

React is a JavaScript library for building user interfaces.

A querystring parser that supports nesting and arrays, with a depth limit

Higher-order functions and common patterns for asynchronous code

Babel compiler core.