prox part 3 — effect abstraction and ZIO
Intro
The first post introduced the prox library and demonstrated the advanced type level programming techniques it uses. Then in the second part of this series, we experimented with replacing the streaming library from fs2 to Akka Streams.
In both cases, the library used cats-effect for describing side effects. But it did not really take advantage of cats-effect ‘s effect abstraction: it explicitly defined everything to be a computation in, cats-effect’s implementation of describing effectful computations.
But we can do better! By not relying on IO
but the various type classes the cats-effect library provides we can make prox work with any kind of effect library out of the box. One such example is ZIO.
Effect abstraction
Let’s see an example of how IO
used to be used in the library! The following function is in the Start
type class, and it starts a process or piped process group:
We can observe two things here:
- The function returns an effectful computation in
IO
- An implicit context shifter is needed by the implementations which are calling some streaming functions needing it.
To make it independent of the effect library implementation we have to get rid of IO
and use a generic type instead, let's call it F
:
Beside using F
instead of IO
everywhere we also have a new requirement, our context type (F
) have to have an implementation of the Concurrent
type class.
Cats-effect defines a hierarchy of type classes to deal with effectful computations. At the time of writing it looks like this:
Read the official documentation for more information.
Prox is based on the ProcessNode
type which has two implementations, a single Process
or a set of processes piped together to a PipedProcess
. Because these types store their I/O redirection within themselves, they also have to be enriched with a context type parameter.
For example Process
will look like this:
The context parameter (F
) is needed because the input source and output target are all representing effectful code such as writing to the standard output, reading from a file, or passing data through concurrent streams.
Let’s see some examples of how the abstract types of cats-effect can be used to describe the computation when we cannot rely on IO
itself!
The most basic operation is to delay the execution of some code that does not use the effect abstractions. This is how we wrap the Java process API, for example.
While with the original implementation of prox it was done by using the IO
constructor:
with an arbitrary F
we only need to require that it has an implementation of the Sync
type class:
and then use the delay
function:
Similarly the Concurrent
type class can be used to start a concurrent computation on a fiber:
Type level
This would be it — except that we need one more thing because of the type level techniques described in the first post.
To understand the problem, let’s see how the output redirection operator works. It is implemented as an extension method on the ProcessNode
type:
This extension method basically just finds the appropriate type class implementations and then call it to alter the process node to register the output redirection:
- we are redirecting the output of
processNode
(of typePN
) toto
(of typeTo
) target
is theCanBeProcessOutputTarget
implementation, containing the actual code to set up the redirectionredirectOutput
is the process node type specific implementation of theRedirectOutput
interface, knowing how to set up the redirection of aProcess
or aPipedProcess
This code would compile, but we won’t be able to use it. For example for the following code:
It fails with not being able to resolve the implicits correctly. The exact error, of course, depends much on the context but one example for the above line could be:
This does not really help understanding the real problem though. As we have seen earlier, in this library the Process
types have to be parameterized with the context as well because they store their redirection logic within themselves. That's why we specify it explicitly in the example to be IO
: Process[IO](...)
. What we would expect is that by tying F[_]
to IO
at the beginning, all the subsequent operations such as the >
redirection would respect this and the context gets inferred to be IO
everywhere in the expression.
The compiler cannot do this. If we check the definition of >
again, you can see that there is no connection expressed between the type PN
(the actual process node type) and F
which is used as a type parameter for the implicit parameters.
The fix is to link the two, and we have a technique exactly for this that I described earlier: the aux pattern.
First let’s write some code that, in compile-time, can “extract” the context type from a process node type:
Both Process
and PipedProcess
have the context as their first type parameter. By creating the ContextOf
type class and the corresponding Aux
type we can extend the >
operator to require such a connection (a way to get a F[_]
context out of a type PN
) in compile-time, and with the aux pattern it unifies the type parameters and the context type gets chained through all the subsequent calls as we desired:
ZIO
Now that everything is in place, we can try out whether prox is really working with other effect libraries such as ZIO.
ZIO has a compatibility layer for cats-effect. It’s the implementation of the type classes cats-effect provides. It is in an extra library called zio-interop-cats.
For running processes with prox we can use the following variants of the ZIO
type:
RIO[-R, +A]
which is an alias forZIO[R, scala.Throwable, A]
- or
Task[A]
which is an alias forZIO[scala.Any, scala.Throwable, A]
if we don't take advantage of the environment parameterR
.
This, in fact, assuming the correct context only means switching IO
to RIO
or Task
in the type parameter for Process
:
A nice way to have everything set up for this is to use the interop library’s trait as an entrypoint for the application.
This brings all the necessary implicits in scope and requires you to implement the following function as the entry point of the application: