spyoungtech/grequests

★ 4,577PythonAudience · developerComplexity · 2/5Setup · easy

Mindmap

mindmap
  root((grequests))
    What it does
      Send many requests at once
      Collect results together
      Stream as they arrive
    Tech stack
      Python
      Gevent
      Requests library
    Use cases
      Bulk URL fetching
      Parallel API calls
      Response streaming
    Audience
      Python developers
      API consumers

mindmap root((grequests)) What it does Send many requests at once Collect results together Stream as they arrive Tech stack Python Gevent Requests library Use cases Bulk URL fetching Parallel API calls Response streaming Audience Python developers API consumers

Click or tap to explore — scroll the page freely

Things people build with this

USE CASE 1

Fetch 50 URLs simultaneously instead of waiting for each one in sequence.

USE CASE 2

Call multiple APIs at once and collect all their responses together.

USE CASE 3

Stream responses as they arrive for faster processing of large batches.

Tech stack

PythonGeventRequests

Getting it running

Difficulty · easy Time to first run · 5min

Import grequests before the requests library to avoid concurrency bugs with Gevent patching.

License details not mentioned in the explanation.

In plain English

GRequests is a Python library that lets you send many web requests at the same time instead of one at a time. Normally, when a Python program fetches a web page or calls an API, it waits for the response before moving on to the next request. If you need to fetch 50 URLs, you wait for the first one to finish, then the second, and so on. GRequests changes this by using a library called Gevent to fire off all the requests at once and collect the results when they are ready. The interface mirrors the popular Requests library closely, so anyone already familiar with how Requests works can switch to GRequests with minimal changes. Instead of calling requests.get() directly, you create a list of request objects and pass them to grequests.map(), which sends them all simultaneously and returns a list of responses in the same order as the inputs. Failed requests return None in the results list, and you can supply an optional error handler function that runs whenever a request fails. For cases where you want to process responses as they arrive rather than waiting for all of them, the library provides an imap() function that returns a generator. Responses come back in whatever order they finish, not the order they were sent, which can be faster for large batches. An enumerated version of imap is also available if you need to know which original request each response corresponds to. One practical note in the README: because Gevent works by patching Python's standard networking code at startup, GRequests should be imported before the Requests library in your code, or you may encounter subtle concurrency bugs. Installation is via pip and the package is available on PyPI.

Copy-paste prompts

Prompt 1

Using grequests in Python, write code that fetches these 10 URLs simultaneously and prints each response's status code: [list your URLs here]

Prompt 2

Show me how to use grequests.imap() to process API responses as they arrive, with an error handler that logs any failed requests.

Prompt 3

Convert this sequential requests.get() loop into a parallel grequests.map() call: [paste your loop here]

Prompt 4

Write a Python script using grequests that calls a REST API endpoint for each item in a list, handles errors gracefully, and returns results in the original order.

Open on GitHub → Explain another repo

← spyoungtech on gitmyhub — every repo by this author, as a profile.

Verify against the repo before relying on details.