This is the last release that will support Python 2.7. Thanks to the many contributors that made this release possible!
- Added support for Python 3.8 and 3.9.
StatefulBrowserhas new properties
url, which can be used in place of the methods
get_urlrespectively (e.g. the new
x.pageis equivalent to
x.get_current_page()). These methods may be deprecated in a future release. [#175]
StatefulBrowser.formwill raise an
AttributeErrorinstead of returning
Noneif no form has been selected yet. Note that
Nonefor backward compatibility.
- Changes in official python version support: added 3.7 and dropped 3.4.
- Added ability to submit a form without updating
submit_selected(..., update_state=False). This means you get a response from the form submission, but your browser stays on the same page. Useful for handling forms that result in a file download or open a new tab.
- Improve handling of form enctype to behave like a real browser. [#242]
typeattributes are no longer required to be lowercase. [#245]
- Form controls with the
disabledattribute will no longer be submitted to improve compliance with the HTML standard. If you were relying on this bug to submit disabled elements, you can still achieve this by deleting the
disabledattribute from the element in the
Formobject directly. [#248]
- When a form containing a file input field is submitted without choosing a file, an empty filename & content will be sent just like in a real browser. [#250]
<option>tags without a
valueattribute will now use their text as the value. [#252]
- The optional
download_linkwas fixed so that it is no longer ignored. [#256]
- Allow duplicate submit elements instead of raising a LinkNotFoundError. [#264]
Our thanks to the many new contributors in this release!
This release focuses on fixing bugs related to uncommon HTTP/HTML scenarios and on improving the documentation.
- Constructing a
Forminstance from a
bs4.element.Tagwhose tag name is not
formwill now emit a warning, and may be deprecated in the future. [#228]
- Breaking Change:
LinkNotFoundErrornow derives from
BaseException. While this will bring the behavior in line with most people’s expectations, it may affect the behavior of your code if you were heavily relying on this implementation detail in your exception handling. [#203]
- Improve handling of
buttonsubmit elements. Will now correctly ignore buttons of type
resetduring form submission, since they are not considered to be submit elements. [#199]
- Do a better job of inferring the content type of a response if the
Content-Typeheader is not provided. [#195]
- Improve consistency of query string construction between MechanicalSoup and web browsers in edge cases where form elements have duplicate name attributes. This prevents errors in valid use cases, and also makes MechanicalSoup more tolerant of invalid HTML. [#158]
StatefulBrowser.refresh()to reload the current page with the same request. [#188]
StatefulBrowser.submit_selected()and the new
StatefulBrowser.download_linknow sets the
Referer:HTTP header to the page from which the link is followed. [#179]
- Added method
StatefulBrowser.download_link, which will download the contents of a link to a file without changing the state of the browser. [#170]
Browser.select_formcan now be a bs4.element.Tag in addition to a CSS selector. [#169]
StatefulBrowser.submit_selectedaccept a larger number of keyword arguments. Arguments are forwarded to requests.Session.request. [#166]
We do not rely on BeautifulSoup’s default choice of HTML parser. Instead, we now specify
lxmlas default. As a consequence, the default setting requires
lxmlas a dependency.
Python 2.6 and 3.3 are no longer supported.
The GitHub URL moved from https://github.com/hickford/MechanicalSoup/ to https://github.com/MechanicalSoup/MechanicalSoup. @moy and @hemberger are now officially administrators of the project in addition to @hickford, the original author.
We now have a documentation site: https://mechanicalsoup.readthedocs.io/. The API is now fully documented, and we have included a tutorial, several more code examples, and a FAQ.
StatefulBrowser.select_formcan now be called without argument, and defaults to
"form"in this case. It also has a new argument,
nr(defaults to 0), which can be used to specify the index of the form to select if multiple forms match the selection criteria.
We now use requirement files. You can install the dependencies of MechanicalSoup with e.g.:
pip install -r requirements.txt -r tests/requirements.txt
Formclass was restructured and has a new API. The behavior of existing code is unchanged, but a new collection of methods has been added for clarity and consistency with the
check(checkboxes are handled differently by default)
Form.print_summarymethod allows you to write
browser.get_current_form().print_summary()to get a summary of the fields you need to fill-in (and which ones are already filled-in).
Formclass now supports selecting multiple options in a
- Checking checkboxes with
browser["name"] = ("val1", "val2")now unchecks all checkbox except the ones explicitly specified.
StatefulBrowser.opennow reset __current_page to None when the result is not an HTML page. This fixes a bug where __current_page was still the previous page.
- We don’t error out anymore when trying to uncheck a box which
doesn’t have a
Form.new_controlnow correctly overrides existing elements.
- The testsuite has been further improved and reached 100% coverage.
- Tests are now run against the local version of MechanicalSoup, not against the installed version.
Browser.add_soupwill now always attach a soup-attribute. If the response is not text/html, then soup is set to None.
<input type=text ...>element instead of an
<input type=input ...>.
- Browser and StatefulBrowser can now be configured to raise a LinkNotFound exception when encountering a 404 Not Found error. This is activated by passing raise_on_404=True to the constructor. It is disabled by default for backward compatibility, but is highly recommended.
- Browser now has a __del__ method that closes the current session when the object is deleted.
- A Link object can now be passed to follow_link.
- The user agent can now be customized. The default includes MechanicalSoup and its version.
- There is now a direct interface to the cookiejar in *Browser classes ((set|get)_cookiejar methods).
- This is the last MechanicalSoup version supporting Python 2.6 and 3.3.
- We used to crash on forms without action=”…” fields.
- The choose_submit method has been fixed, and the btnName argument of StatefulBrowser.submit_selected is now a shortcut for using choose_submit.
- Arguments to open_relative were not properly forwarded.
- The testsuite has been greatly improved. It now uses the pytest API (not only the pytest launcher) for more concise code.
- The coverage of the testsuite is now measured with codecov.io. The results can be viewed on: https://codecov.io/gh/hickford/MechanicalSoup
- We now have a requires.io badge to help us tracking issues with dependencies. The report can be viewed on: https://requires.io/github/hickford/MechanicalSoup/requirements/
- The version number now appears in a single place in the source code.
see Git history, no changelog sorry.