Hultnér Technologies AB | @ahultner | Blog | Slides, foss-north 2019 | Nordisk Python Community | GitHub
Property based testing in Python using Hypothesis.
# Minimal usage example
@given(
st.integers(),
st.integers()
)
def test_ints_are_commutative(x, y):
assert x y == y x
# Define special edge cases with @example,
# especially useful for regression testing
@given(st.integers(), st.integers())
@example(-1, 1)
def test_add(a, b):
assert add(a, b) == a bProperty Based Testing in Python with Hypothesis - Alexander Hultnér, Episode 107
Hypothesis is the Python tool used for property based testing.
Hypothesis claims to combine "human understanding of your problem domain with machine intelligence to improve the quality of your testing process while spending less time writing tests."
In this episode Alexander Hultnér introduces us to property based testing in Python with Hypothesis.
Some topics covered:
- What is property based testing
- Thinking differently for property based testing
- Using hypothesis / property based testing in conjunction with normal testing
- Failures saved and re-run
- What parts of development/testing is best suited for hypothesis / property based testing
- Comparing function implementations
- Testing against REST APIs that use Open API / Swagger with schemathesis
- Changing the number of tests in different test environments
- System, integration, end to end, and unit tests
Did you ever miss that corner case bug? Maybe it was a negative integer, strange timezone conversion behaviour, off by one error or something entirely else. These subtle bugs are often hard to catch and are easily missed in test cases. You like me have probably ran into plenty of code utilising only happy path testing, only to later discover subtle bugs which are easily fixed once pointed out. This is where property based testing comes into the picture.
In this talk I will focus on a wonderful Python library called Hypothesis but the concepts apply to other languages as well. Hypothesis is based on the same concept as the famous QuickCheck library for Haskell, which in turn have been ported a large number of languages. Hypothesis uses a wide range of input to find edge cases that you could otherwise easily miss, once it finds these cases it narrows down the input to the minimal breaking example to provide failures which are easier to understand.
Audience level: Intermediate
Speaker: Alexander Hultnér
Test Fast, Fix More – Property based in Python testing with Hypothesis (youtube video)
Did you ever miss that corner case bug? Maybe it was a negative integer, strange timezone conversion behaviour, off by one error or something entirely else. These subtle bugs are often hard to catch and are easily missed in test cases. You like me have probably ran into plenty of code utilising only happy path testing, only to later discover subtle bugs which are easily fixed once pointed out. This is where property based testing comes into the picture.
In this talk I will focus on a wonderful Python library called Hypothesis but the concepts apply to other languages as well. Hypoethesis is based on the same concept as the famous QuickCheck library for Haskell, which in turn have been ported a large number of languages. Hypothesis uses a wide range of input to find edge cases that you could otherwise easily miss, once it finds these cases it narrows down the input to the minimal breaking example to provide failures which are easier to understand.
What's the requirements for using Hypothesis?
Install it via pip install hypothesis check out the quickstart guide
Do I need to rewrite all my tests?
Absolutely not, I'd encourage to use property based testing as a complement to your current testing
and start experimenting with it on critical parts.
Is this the same thing as a fuzzer?
Not quite but similar concepts, I like to think of property-based testing as structured fuzzing.