Announcing ciao-lwt: A Library for Migrating from Lwt to Eio

The I/O library Eio, which uses effects and direct-style concurrency, was released in 2024. Since then, users have seized the opportunity to test it with their own projects and several OCaml devs have ported applications to Eio.

Now, with a new tool called ciao-lwt, users can automate part of the migration process from Lwt to Eio. One of our engineers, Jules Aguillon, has been working on the tool and using it to migrate an Lwt code project to Eio. This post will introduce you to ciao-lwt, where you can try it, and what limitations you should expect.

Why Would I Switch to Eio?

Ultimately, the concurrency library you choose comes down to a matter of taste, but Eio has some nice characteristics that you may find worth the switch. Since it is direct-style, Eio does not require you use a monad for concurrency, which gets rid of the so-called ‘[function colouring problem](’. The resulting code is faster, less complex, and has some nice security capabilities. You can read our blog post on Eio 1.0 for more context.

Ciao-Lwt

The library contains a collection of tools for translating an Lwt library into Eio code. Lwt marks concurrent code and non-concurrent (sync and async) code using bind operators or bindings, and functions must be explicitly marked as sync or async for the program to run. These are the ‘bind’ and ‘map’ operators, including Lwt.bind, Lwt.map, let*, let+, as well as the infix operators >>= and >>|.

The first step in turning Lwt into Eio code is to get rid of the bind operators. They weave through every part of Lwt code and are time-intensive to remove one by one, but ciao-lwt can automate the process and remove the bindings for you. However, the library has some limitations, which are important to note and will be explained in more detail below.

As it stands, the library contains the following tools:

• lwt-ppx-to-let-syntax: Removes instances of lwt_ppx and replaces them with Lwt library function calls,
• lwt_lint: Finds implicit forks in Lwt code,
• lwt-log-to-logs: Rewrites files containing Lwt_logand migrates them to use Logs,
• lwt-to-direct-style: Finds and rewrites Lwt and other Lwt modules, turning them into Eio code instead.

How Do I Try It?

To get started with ciao-lwt, the first thing to do is visit the repo, and install the tools in the opam switch you’re using to build your projects.

To make reviewing the change easier, make sure your code is formatted. The tool will entirely reformat the file it touches, which may make actual changes harder to see.

The first step is to remove any use of lwt_ppx (for example the let%lwt syntax):

lwt-ppx-to-let-syntax .
dune fmt # Remove formatting changes created by the tool

This operation is purely syntactical, the tool simply walks the given directory tree and parses every .ml files it finds, updating the files that contain usages of lwt_ppx.

Before running the next tools, try eliminating common cause of implicit forks:

lwt-lint .

This operation is also purely syntactical. The tool warns about every occurrence of let _ = .. and ignore that doesn’t have a type annotation. This helps you find cases where an Lwt promise is ignored. To silence each warning, add a type annotation, for example: let _ : my_t = .. and ignore (.. :my_t).

If you use Lwt_log, you can migrate to Logs easily with:

dune build @ocaml-index # Build the index (required)
ciao-lwt to-logs --migrate .
dune fmt # Remove formatting changes created by the tool

This tool works similarly to ciao-lwt to-eio described below. It is provided as a separate command because your program will likely work like before, it lets you reviewe this step separately, and it simplifies the next step.

Finally, migrate to Eio:

dune build @ocaml-index # Build the index (required)
ciao-lwt to-eio --migrate .
dune fmt # Remove formatting changes created by the tool

This operation migrates the common uses of Lwt but the transition is not complete.

Lastly, something to bear in mind is that for all of cial-lwt's tools, Merlin's index is used to locate every use of Lwt.

Limitations & Considerations

Ciao-lwt is still considered experimental and a work-in-progress, which you should bear in mind when you try it out. Your feedback and input is very welcome and will help the team improve the tools.

It sounds obvious, but as a promise-based concurrency library, Lwt creates a lot of promises. Everything that is concurrent in Lwt is a promise; it specifies actions that will happen at a later time. Some of these promises are so-called ‘implicit promises’ which are created using those same bindings we mentioned earlier.

Eio doesn’t use promises; they are considered too resource-intensive, but instead uses forks or fibre calls to specify the order in which operations are to be performed when necessary. The problem is that since Lwt uses bindings to create promises and also to label async and sync code, simply removing all the bindings also removes the promises. In cases where a promise exists not simply because it was written in Lwt style but because it needed to be there, ciao-lwt will break that promise.

To partially remedy this problem, Jules created the lwt_lint tool, which finds implicit promises in Lwt code that are not there simply because they are promise-style. It warns users about values bound to let_ or passed to ignore that do not have a type annotation, which helps users find ignored Lwt threads.

When you use ciao-lwt, it’s important to be aware of how removing the bindings that ‘label’ async or sync code in Lwt projects can affect promises. Using lwt_lint to find promises that need to be translated into forks or fibre calls in direct-style Eio code is helpful, but not foolproof. You will need to be aware that your code could break in this way, and look out for it when you use the library.

Until Next Time

Tarides remains committed to creating new tools that make new and old workflows easier. We hope ciao-lwt proves useful to you, and appreciate any feedback you have to share.

You can connect with us on Bluesky, Mastodon, Threads, and LinkedIn or sign up for our mailing list to stay updated on our latest projects. We look forward to hearing from you!