jmcarp/robobrowser

RoboBrowser is a Python library for automating web browsing without opening an actual browser. You can use it to open URLs, click links, fill in forms, tick checkboxes, upload files, and navigate back through page history. The main use case the README describes is interacting with websites that have no public API, where you would otherwise have to do things manually in a browser window. Under the hood it combines two existing Python libraries. Requests handles the actual HTTP communication, including session cookies, custom headers, and proxy settings. BeautifulSoup parses the HTML of each page so you can search it using CSS selectors or regular expressions to find links, forms, and page elements. RoboBrowser exposes both libraries through one interface, so you can inspect session cookies and search the page structure from the same object. Form handling is a notable focus. The library lets you find forms by CSS class, action URL, or index, inspect their current field values, set text fields and checkboxes, and submit. Checkboxes behave like lists: you set which options are checked by passing a list of values or labels, and the library raises an error if you try to set a value that does not exist in the form. File upload inputs are also supported. The README includes short code examples for each scenario: searching a lyrics site, inspecting session cookies on a GitHub page, signing up on a form, toggling checkboxes, and uploading a file. The library requires Python 2.6+ or 3.3+ and is licensed under the MIT license.