Cookbooks
Haskell
Chapter 11. HTML and the Web
Downloading a web page

Haskell:
Downloading a web page

How to:

Let’s roll with a straightforward example using Haskell’s http-conduit library. First, install it using cabal install http-conduit. Then:

import Network.HTTP.Conduit -- The main network library
import qualified Data.ByteString.Lazy as L -- We'll need Lazy ByteStrings

-- Function to download a web page
downloadPage :: String -> IO L.ByteString
downloadPage url = simpleHttp url

main :: IO ()
main = do
    -- Use the function to download a page
    content <- downloadPage "http://example.com"
    -- Do something with the content, like printing it
    L.putStr content

Running this, you’ll see the HTML of http://example.com on your screen.

Deep Dive

HTTP requests in Haskell haven’t always been this neat. Older libraries like HTTP required more boilerplate code. With http-conduit, the complexity is abstracted away.

Other methods exist, like the wget command in a shell script or Python’s requests library. But these aren’t always as efficient or expressive in Haskell’s functional setting.

Under the hood, http-conduit uses a Manager to handle connection pooling and Keep-Alive for HTTP1.1, making it more efficient for multiple requests.

See Also

Parsing HTML