pipes-4.0 which greatly simplifies the types and API of the
pipes ecosystem. For people new to
pipes is a compositional streaming library that decouples streaming programs into simpler, reusable components.
The purpose behind
pipes is to simplify effectful stream programming for both library authors and application writers. Library authors can package streaming components into a highly reusable interface, and application writers can easily connect together streaming components to build correct, efficient, and low-memory streaming programs. For example, the following program connects three reusable components to stream lines from standard input to standard output until the user types
import Pipes import qualified Pipes.Prelude as P main = runEffect $ P.stdinLn >-> P.takeWhile (/= "quit") >-> P.stdoutLn
pipes distinguishes itself from other stream programming libraries in three main ways:
- Insistence on elegance, symmetry, and theoretical purity
- Careful attention to correctness, documentation, and detail
- Emphasis on a high power-to-weight ratio
- The Hackage page
- The official tutorial
- The Github repository
- The Haskell wiki page
- The haskell-pipes mailing list
This release was made possible due to the suggestions and contributions of many people and I want to give special mention to several people:
Renzo Carbonara, who is the largest contributor of downstream libraries, building and maintaining
pipes-aeson. He also provided lots of useful feedback on design proposals because of his experience maintaining these libraries.
Ben Gamari, who contributed
Oliver Charles who contributed to the design of the new
pipes-parselibrary in the process of developing the
Csernik Flaviu Andrei, who contributed the complete benchmark suite for
Merijn Verstraaten, who contribute the new
Gergely Risko, who fixed a concurrency bug in
Mihaly Barasz, who contributed the complete test suite for
Tony Day who helped automate the
pipesecosystem and contributed lots of useful feedback on documentation.
Aleksey Khudyakov first proposed the idea to remove the old proxy transformer system and outsource the same functionality to monad transformers in the base monad. This change alone accounted for an almost 60% reduction in the library size and the greatest simplification of the types.
Johan Tibell proposed the initial idea to provide a simpler unidirectional subset of the API by default. This removed the warts that bidirectionality introduced.
Florian Hofmann whose work on
pipes-eepled to the discovery of an
Arrowinstance for push-based pipes.
Oliver Batchelor, whose work on integrating
pipeswith Cloud Haskell improved the design of
Also, I would like to also thank everybody who provided feedback on the library and its documentation and also contributed code.
People familiar with
pipes will notice that the biggest change to the library is the elimination of the proxy transformer system. This was made possible by an insight of Aleksey Khudyakov that the proxy transformers were isomorphism to monad transformers in the base monad if you ignored their ability to be unwrapped before the
Proxy layer. I later discovered how to unwrap these base monad transformers while preserving the
Proxy layer, which made possible the complete elimination of the proxy transformer system.
This had the largest impact on simplifying the API:
The number of exported functions dropped to approximately 1/3 of the original size (from about 300+ to 100+)
The number of modules dropped to 1/3 of the original size (from 18 to 6)
ptype parameter in type signatures disappeared, along with the
Proxytype class, which became the concrete
Proxytype (which was the old
No need for
The next most important change was a simplification of the API to a unidirectional subset which is the new default. This fixed several warts of the previous API:
No more gratuitous
The pipe monad and category now overlap
Polymorphic type synonyms can now be used to simplify the types
The original bidirectional functionality still remains intact within the
Pipes.Core module. The only difference is that it is not exported by default.
The next important change was the realization that bind in the
respond Category (i.e.
(//>)) was exactly equivalent to a
for loop, so the unidirectional API now uses
for as a synonym for
(//>) and produces really beautiful
Other important syntactic changes:
The unidirectional API uses
respondlike it was back in
The unidirectional API uses
requestlike it was back in
(>->)is the unidirectional pull-based composition, instead of bidirectional composition
Pipes.Prelude has also changed to remove the suffix from all utilities, but is no longer re-exported from the main
The downstream libraries have been updated as well to use the
pipes-4.0 API and several of these now have much simpler APIs, too, particularly
pipes-safe. I will discuss these libraries in separate library-specific posts later on.
This release is intended to be the last major version bump. The next development priorities are:
Stabilize the core
Improve distribution by packaging up
pipesfor several package managers
Continue to build out the
pipesecosystem, particularly dedicated
The long-term goal is to get
pipes into the Haskell platform once the API has proven itself stable and the ecosystem matures.
People interested in learning more about
pipes or contributing to development can join the official mailing list.