Quickcheck
QuickCheck
Quickcheck is a type-driven testing with random test case generation.
QuickCheck is a great tool:
A domain-specific language for writing properties.
Test data is generated automatically and randomly.
Another domain-specific language to write custom generators.
You should use it.
The smallcheck, hedgehog and HUnit libraries are also worth checking out.
However, keep in mind that writing good tests still requires training, and that tests can have bugs, too.
Call quickCheck function to test the properties.
quickCheck :: Testable prop => prop -> IO ()
The argument of quickCheck is a property of type prop .
The only restriction on the type prop is that it is in the Testable type class.
Testable properties usually are functions (with arbitrarily many arguments) resulting in a Bool
When executed, quickCheck prints the results of the test to the screen – hence the IO () result type.
So far, all our properties have been of type [Int] -> Bool :
code:test.hs
sortPreservesLength :: Int -> Bool sortEnsuresSorted :: Int -> Bool sortPermutes :: Int -> Bool When used on such properties, QuickCheck generates random integer lists and verifies that the result is True.
If the result is True for 100 cases, this success is reported in a message.
If the result is False for a case, the test case triggering the result is printed.
Analyzing the test data
collect :: (Testable prop, Show a) => a -> prop -> Property
The function collect gathers statistics about test cases. This information is displayed when a test passes
The type property
The type Property is QuickCheck-specific. It holds more structural information about a property than a plain Bool ever could.
Like Bool, a Property is testable, so for us, not much changes.
code:tesable.hs
instance Testable Property where ...
Implication
Some of the generated test data does not fulfill the requirements needed for testing.
The solution is to use the QuickCheck implication operator :
code:operator.hs
(==>) :: (Testable prop) => Bool -> prop -> Property
We see Property again – this type allows us to encode not only True or False but also to reject the test case.
code:imply.hs
iPO :: Int -> Int -> Property iPO x xs = sorted xs ==> sorted (insert x xs)
Now, lists that are not sorted are discarded and do not contribute towards the goal of 100 test cases.
Custom Generators
Generators belong to an abstract data type Gen.
We can define our own generators using another domain-specific language. The default generators for datatypes are specified by defining instances of class Arbitrary
code:defineArbitrary.hs
class Arbitrary a where
arbitrary :: Gen a
Think of Gen a as an abstract set of information on how to produce values of type a randomly
Running generators
QuickCheck has internal functions to extract random values from generators.
For end users, two debugging functions are offered
code:sample.hs
sample :: Show a => Gen a -> IO ()
sample' :: Show a => Gen a -> IO a These produce a number of random values generated by the given Gen a, and print them in the case of sample, or return them in the case of sample'.
Build new generators
QuickCheck includes a library for the construction of new generators:
code:genbuilder.hs
choose :: Random a => (a, a) -> Gen a
sized :: (Int -> Gen a) -> Gen a
Below is an example of building a generator for datatype Person
code:example.hs
data Person = Person
{ pName :: !String
, pAge :: !Int
} deriving Show
instance Arbitrary Person where
arbitrary = do
age <- choose (10, 100)
pure $ Person name age
Modifiers
The module Test.QuickCheck.Modifiers defines a number of newtype wrappers. These can be used so that the developer does no need to implement them every time they use quickcheck.
code:modifiers.hs
newtype Positive a = Positive a
newtype NonNegative a = NonNegative a
newtype NonZero a = NonZero a
newtype NonEmptyList a = NonEmptyList a newtype OrderedList a = OrderedList a iPO :: Int -> OrderedList Int -> Bool
iPo x (Ordered xs) = sorted (insert x xs)
Sample code
code:testcode.hs
module Main where
import Control.Exception
import Data.List (permutations)
import Test.Hspec
import Test.Hspec.QuickCheck
import Test.QuickCheck
import Sort
--------------------------------------------------------------------------
-- Properties:
sortPreservesLength :: Int -> Bool sortPreservesLength xs = length xs == length (sort xs)
preserves :: Eq a => (t -> t) -> (t -> a) -> t -> Bool
(f preserves p) x = p x == p (f x)
sortPreservesLength' :: Int -> Bool sortPreservesLength' = sort preserves length
idPreservesLength :: Int -> Bool idPreservesLength = id preserves length
--------------------------------------------------------------------------
-- Specifying sortedness:
sorted :: Int -> Bool -- not correct sorted [] = True
sorted (x : y : ys) = x <= y && sorted (y : ys)
(f ensures p) x = p (f x)
sortEnsuresSorted :: Int -> Bool sortEnsuresSorted = sort ensures sorted
f permutes xs = f xs elem permutations xs
sortPermutes :: Int -> Bool sortPermutes xs = sort permutes xs
--------------------------------------------------------------------------
-- Other properties:
appendLength :: a -> a -> Bool appendLength xs ys = length xs + length ys == length (xs ++ ys)
plusIsCommutative :: Int -> Int -> Bool
plusIsCommutative m n = m + n == n + m
takeDrop :: Int -> Int -> Bool takeDrop n xs = take n xs ++ drop n xs == xs
dropTwice :: Int -> Int -> Int -> Bool dropTwice m n xs = drop m (drop n xs) == drop (m + n) xs
lengthEmpty :: Bool
lengthEmpty = length [] == 0
wrong :: Bool
wrong = False
--------------------------------------------------------------------------
-- Generators:
mkSorted [] = []
mkSorted (x : y : ys) = x : mkSorted (x + abs y : ys)
-- Can you see a potential problem with mkSorted?
genSorted = fmap mkSorted arbitrary
implies :: Bool -> Bool -> Bool
implies x y = not x || y
insertPreservesSortedNaive :: Int -> Int -> Bool insertPreservesSortedNaive x xs =
sorted xs implies sorted (insert x xs)
insertPreservesSortedNaive' :: Int -> Int -> Property insertPreservesSortedNaive' x xs =
sorted xs ==> sorted (insert x xs)
insertPreservesSorted :: Int -> Property
insertPreservesSorted x = forAll genSorted (\ xs -> sorted xs ==> sorted (insert x xs))
insertPreservesSorted' :: Int -> OrderedList Int -> Bool
insertPreservesSorted' x (Ordered xs) = sorted (insert x xs)
--------------------------------------------------------------------------
-- Haskell Program Coverage:
main :: IO ()
main = hspec $ do
describe "sort" $ do
it "preserves length" $ property sortPreservesLength
prop "ensures sorted" sortEnsuresSorted
describe "other properties" $ do
prop "take and drop" takeDrop
prop "plus is commutative" plusIsCommutative
it "head [] should fail" $ evaluate (head []) shouldThrow anyErrorCall
Useful links