Browser Automation

Hrequests supports both Firefox and Chrome browsers, headless and headful sessions, & browser extensions

Browser support table

Chrome supports both Manifest v2/v3 extensions. Firefox only supports Manifest v2 extensions.

Only Firefox supports CloudFlare WAFs.

Browser	MV2	MV3	Cloudfare WAFs
Firefox	✅	❌	✅
Chrome	✅	✅	❌

---

Usage

You can spawn a BrowserSession instance by calling it:

>>> page = hrequests.BrowserSession()  # headless=True by default

Parameters

Parameters:
    headless (bool, optional): Whether to run the browser in headless mode. Defaults to True.
    session (hrequests.session.TLSSession, optional): Session to use for headers, cookies, etc.
    resp (hrequests.response.Response, optional): Response to update with cookies, headers, etc.
    proxy (str, optional): Proxy to use for the browser. Example: http://1.2.3.4:8080
    mock_human (bool, optional): Whether to emulate human behavior. Defaults to False.
    browser (Literal['firefox', 'chrome'], optional): Generate useragent headers for a specific browser
    os (Literal['win', 'mac', 'lin'], optional): Generate headers for a specific OS
    extensions (Union[str, Iterable[str]], optional): Path to a folder of unpacked extensions, or a list of paths to unpacked extensions

By default, BrowserSession returns a Chrome browser.

To create a Firefox session, use the chrome shortcut instead:

>>> page = hrequests.firefox.BrowserSession()

BrowserSession is entirely safe to use across threads.

---

Render an existing Response

Responses have a .render() method. This will render the contents of the response in a browser page.

Once the page is closed, the Response content and the Response's session cookies will be updated.

Simple usage

Rendered browser sessions will use the browser set in the initial request.

You can set a request's browser with the browser parameter in the hrequests.get method:

>>> resp = hrequests.get('https://example.com', browser='chrome')

Or by setting the browser parameter of the hrequests.Session object:

>>> session = hrequests.Session(browser='chrome')
>>> resp = session.get('https://example.com')

Example - submitting a login form:

>>> session = hrequests.Session(browser='chrome')
>>> resp = session.get('https://www.somewebsite.com/')
>>> with resp.render(mock_human=True) as page:
...     page.find('.input#username').type('myuser')
...     page.find('.input#password').type('p4ssw0rd')
...     page.click('#submit')
# `session` & `resp` now have updated cookies, content, etc.

Or, without a context manager

>>> session = hrequests.Session(browser='chrome')
>>> resp = session.get('https://www.somewebsite.com/')
>>> page = resp.render(mock_human=True)
>>> page.find('.input#username').type('myuser')
>>> page.find('.input#password').type('p4ssw0rd')
>>> page.click('#submit')
>>> page.close()  # must close the page when done!

The mock_human parameter will emulate human-like behavior. This includes easing and randomizing mouse movements, and randomizing typing speed. This functionality is based on botright.

Parameters

Parameters:
    headless (bool, optional): Whether to run the browser in headless mode. Defaults to False.
    mock_human (bool, optional): Whether to emulate human behavior. Defaults to False.
    extensions (Union[str, Iterable[str]], optional): Path to a folder of unpacked extensions, or a list of paths to unpacked extensions

---

Other ways to create a Browser Session

You can use .render to spawn a BrowserSession object directly from a url:

# Using a Session:
>>> page = session.render('https://google.com')
# Or without a session at all:
>>> page = hrequests.render('https://google.com')

Make sure to close all BrowserSession objects when done!

>>> page.close()