Four Tools for Managing Monorepos

Microservice architecture is a popular way to build software applications. The approach has many advantages, notably ease of scaling and avoidance of blocking issues in the case of a critical error. It’s common for software projects to have multiple repositories (poly-repo) in a microservice architecture, so that software developers can work flexibly in terms of programming languages or tech stack, which are service specific. However, simultaneous use of multiple repositories brings some significant downsides, such as difficulties with setting up and maintaining CI/CD pipelines and challenges in tracking software tasks since multiple repositories can be deployed in a single task. This is where monorepo comes into play.

Monorepo

Monorepo is a software approach that lets you put your application code into a single repository. Monorepo (monorepository) allows every developer on a team to be aware of changes made to the product code, letting them know when breaking changes occur. Monorepo also helps DevOps easily set up CI/CD pipelines since all the codes reside in a single location.

A good monorepo tool should provide a powerful command line interface to help you create the boilerplate code, manage software dependencies, and test support utilities. It should also work well with a number of different frameworks, such as React, Next, Vue, and Angular, so that you can apply it to your existing project. Tools like Lerna, npm, Turborepo, and Rush satisfy these requirements. Let’s check out the advantages and disadvantages of each of these tools so that you can choose the one that best fits your needs.

Lerna

Lerna is a widely used monorepo tool for JavaScript/TypeScript developers, offering a powerful command line interface and graph visualizer. In April 2022, it was announced that Lerna would no longer be maintained. However, just a month later, in May 2022, Lerna was taken over by the Nx team, who have brought speed and caching behavior improvements to Lerna.

Advantages

• Supports versioning for the repository—you can always revert back to an old version if an updated version of the repository doesn’t work.
• Supports running commands concurrently, which allows an impressive speed of command executions.
• Provides a graph visualizer, giving you an overview of the repository code’s architecture.
• Allows caching data to be distributed amongst different CI/CD machines, reducing execution time in CI/CD environment.

Disadvantages

• Lacks support for running only impacted tests (i.e., modified packages).

Example Use Case

To better understand how Lerna works, let’s walk through an example monorepo project with different packages for blog management, movie-api, and BrowserStack tests.

First, clone the example project from this GitHub repository.

git clone https://github.com/cuongld2/lerna-demo-usecase.git
git checkout initBranch

Next, install the current dependencies required from the main project folder:

npm install

To apply Lerna to your project, simply run:

npx lerna@latest init

You should see output similar to the below:

lerna notice cli v6.1.0
lerna info Creating .gitignore
lerna info Updating package.json
lerna info Creating lerna.json
lerna info Creating packages directory
lerna success Initialized Lerna files
lerna info New to Lerna? Check out the docs: https://lerna.js.org/docs/getting-started

Take a closer look at the current folder structure, and you’ll notice that Lerna automatically creates a lerna.json file to manage the project. There’s also a blank packages folder. To let Lerna search for packages you want to use, you must put your project packages inside the packages directory.

mv blog-management browserstack-test movie-api -t packages 

Lerna supports visualizing the relationships between packages in the project. To see visualizations, run:

npx nx graph

You should see output like this:

>  NX   Project graph started at http://127.0.0.1:4211

Open your browser at 127.0.0.1:4211 to see the package information inside your project.

To run tests for all three packages, execute the following command:

npx lerna run test

You should see the output as follows, showing the results of the test:

>  Lerna (powered by Nx)   Running target test for 3 projects

    →    Executing 1/1 remaining tasks...

    ⠴    jest-selenium-browserstack-getstarted:test

    ✔    1/2 succeeded [0 read from cache]
    ✖    1/2 failed

Lerna's official documentation offers more details on using the tool.

npm

npm is a common tool for JavaScript/TypeScript developers since most introductory guides for developers use npm. It lets you initialize the JavaScript/TypeScript project quickly and provides standard build tool features, such as managing dependencies, adding custom scripts, and building new plugins.

Advantages

• Easy to use.
• Many features offered.
• Familiar to most developers.

Disadvantages

• Performance suffers when there are a large number of dependencies.
• Weak security offerings.

To learn more about how npm works, check out npm's official documentation.

Turborepo

Turborepo is another build tool that was created for managing monorepo projects. It was built using Rust programming language, making it extremely fast. Turbo is well suited for a number of projects in JavaScript/TypeScript environments, such as React, Next, and Angular projects.

Advantages

• Extremely fast.
• Offers better security than comparable tools.
• Supports remote caching.
• Allows running tests in watch mode.
• Supports integration with CI tools like GitHub Action or CircleCI, with guided examples available using the turbo.json file.

Disadvantages

• Likely a new tool for developers since it is not commonly used.
• Does not support versioning by default; requires third-party tools like changeset and beachball.

Example Use Case

To understand how Turborepo works, let’s create an example project with one front-end application and one package that defines the application’s user interface components.

First, clone the example project from this GitHub repository:

git clone https://github.com/cuongld2/turborepo-demo.git

Turbo-repo works based on workspaces. You can flexibly configure workspaces that satisfy your needs. In this project, we’ll configure workspaces in the package.json file:

"workspaces": [
    "packages/*",
    "apps/*"
  ]

Using packages/* and apps/*, you can tell Turborepo that all the directories inside packages and apps are workspaces. Set the configuration to be more specific, such as packages/components. This means only the components directory inside the packages directory is designated as Turborepo’s workspace.

One of Turborepo’s great features is the support it offers for deployment pipelines. You can define your pipeline inside the turbo.json file at the main directory of the project:

{
    "$schema": "https://turborepo.org/schema.json",
    "baseBranch": "origin/main",
    "pipeline": {
        "build": {
            "dependsOn": [
                "^build"
            ],
            "outputs": [
                "dist/**"
            ]
        },
        "test": {
            "dependsOn": [
                "^build"
            ],
            "outputs": []
        },
        "lint": {
            "outputs": []
        },
        "dev": {
            "cache": false
        }
    }
}

This pipeline has stages for build, test, lint (checking syntax and code format,) and dev (development purpose without using the cache data.)

To install the dependencies for the project, run yarn from the main directory; yarn is a build tool and can be used with Turborepo to manage monorepo projects efficiently.

npm install -g yarn
yarn

You should see this output:

[4/4] Building fresh packages...
success Saved lockfile.
Done in 26.77s.

Run the whole development pipeline:

yarn run dev

You should see output like this:

yarn run v1.22.19
$ turbo run dev --parallel --no-cache
• Packages in scope: components, frontend
• Running dev in 2 packages
• Remote caching disabled
frontend:dev: Starting the development server...
frontend:dev: ℹ Compiling Webpack
frontend:dev: ✔ Webpack: Compiled successfully in 2.53s
frontend:dev:  DONE  Compiled successfully in 2531ms7:18:49 AM
frontend:dev: 
frontend:dev: 
frontend:dev:   App running at:
frontend:dev:   - Local:   http://localhost:8000 (copied to clipboard)
frontend:dev:   - Network: http://192.168.1.6:8000
frontend:dev:  WAIT  Compiling...7:18:49 AM
frontend:dev: 
frontend:dev: ℹ Compiling Webpack
frontend:dev: ✔ Webpack: Compiled successfully in 98.08ms
frontend:dev:  DONE  Compiled successfully in 102ms7:18:50 AM

Open the browser and navigate to localhost:8000; you should see that the app is up and running:

For details on how to use Turborepo, check out the official Turborepo documentation.

Rush

Rush was created to support parallel builds and manage hundreds of code services in a single repository project. Rush was made—and is still used—by Microsoft for developing their own software products. You can count on Rush when managing a monorepo project that must balance scalability and efficiency.

Advantages

• Avoids broken imports by introducing isolated symlinks.
• Supports pnpm to remove duplicate packages.
• Integrates well with CI pipelines by providing scripts like install-run-rush.js for installing Rust in CI tools and command line options like rush init when working with GitHub Actions.
• Provides watch mode, allowing Rush to spot changes in the source code and alert you as to whether the change will impact existing application features.

Disadvantages

• Has a steep learning curve.

Example Use Case

Let’s take the same situation in our Lerna use case to explore Rush in action.

First, clone this example repository:

git clone https://github.com/cuongld2/demo-rush.git
git checkout init

Run the following command to install Rush:

npm install -g @microsoft/rush

Note: Rush does not support latest version of Node.js yet (version 18,) so you need to use a Node version in this range: >=14.19.0 <15.0.0 || >=16.13.0 <17.0.0.

To initialize the Rush project, run:

rush init

Rush will then automatically generate some files, including rush.json, the main configuration file. There’s also a new directory called common, which stores configuration files for npm, pnpm, yarn, and caching configuration files.

Let’s put three packages inside a new apps folder:

mkdir apps
mv ../blog-management ../movie-api ../browserstack-test -t apps

You also need to tell Rush where to look for these packages. Open the rush.json file and add the following content. (You must also remove all comments inside the rush.json file, since comments are not supported.)

"projects": [
    {
      "packageName": "blog-management",
      "projectFolder": "apps/blog-management"
    },
    {
      "packageName": "movie-api",
      "projectFolder": "apps/movie-api"
    },
    {
      "packageName": "browserstack-test",
      "projectFolder": "apps/browserstack-test"
    }
  ]

To apply these updates, run:

rush update

You should then see this message:

Rush update finished successfully. (33.46 seconds)

Rush's official documentation offers more details about the tool.

Conclusion

The four most popular monorepo tools each have specific strengths and weaknesses. As is usually the case with software, there is no perfect tool; instead, you must pick the best tool for your project’s specific needs.

If you have a new project, don’t forget to check out Amplication. Amplication is an open-source tool that allows you to generate boilerplate code for common functionalities such as authentication, authorization, unit testing, or interacting with databases. With Amplication, you can quickly and efficiently build Node.js applications with built-in best practices.