Overview

Lint open source repositories for common issues.

Installation

Repolinter requires Node.JS >= v10 to function properly. Once Node.JS is installed, you can install Repolinter using npm:

npm install -g repolinter

Linting a Local Repository

Once installed, run the following to lint a directory:

repolinter lint <directory>

The above command will lint <directory> with the local repolinter.json ruleset or the default ruleset if none is found:

repolinter % repolinter lint .
Target directory: <directory>
Lint:
✔ license-file-exists: Found file (LICENSE)
✔ readme-file-exists: Found file (README.md)
✔ contributing-file-exists: Found file (CONTRIBUTING)
✔ code-of-conduct-file-exists: Found file (CODE-OF-CONDUCT)
✔ changelog-file-exists: Found file (CHANGELOG)
...
repolinter % echo $?
0

Linting a Remote Repository

Repolinter also supports linting a git repository using the --git flag. With this flag enabled, the directory input will be interpreted as a git URL which Repolinter will automatically clone into a temporary directory.

repolinter lint -g https://github.com/todogroup/repolinter.git

Formatting the Output

The Repolinter CLI currently supports three output formatting modes:

• Default (also referred to as result)
• JSON
• Markdown

You can switch formatters using the --format flag. An example of using the JSON formatter:

repolinter % repolinter lint --format json .
{"params":{"targetDir":"/Users/nkoontz/Documents/code/repolinter","filterPaths":[],...

An example of using the Markdown formatter:

repolinter % repolinter lint --format markdown .
# Repolinter Report

This Repolinter run generated the following results:
| ❗  Error | ❌  Fail | ⚠️  Warn | ✅  Pass | Ignored | Total |
|---|---|---|---|---|---|
| 0 | 0 | 0 | 15 | 10 | 25 |
...

Limiting Paths

Repolinter supports an allowed list of paths through the --allowPaths option to prevent the accidental linting of build artifacts. These paths must still be contained in the target directory/repository.

repolinter lint --allowPaths ./a/path --allowPaths /another/path

Disabling Modifications

By default Repolinter will automatically execute fixes as specified by the ruleset. If this is not desired functionality, you can disable this with the --dryRun flag.

Ruleset Configuration

Similar to how eslint uses an eslintrc file to determine what validation processes will occur, Repolinter uses a JSON or YAML configuration file (referred to as a ruleset) to determine what checks should be run against a repository. Inside a ruleset, there are two main behaviors that can be configured:

• Rules - Checks Repolinter should perform against the repository.
• Axioms - External libraries Repolinter should use to conditionally run rules.

These combined capabilities give you fine-grained control over the checks Repolinter runs.

Providing a Ruleset

Repolinter will pull its configuration from the following sources in order of priority:

1. A ruleset specified with --rulesetFile or --rulesetUrl
2. A repolint.json, repolinter.json, repolint.yaml, or repolinter.yaml file at the root of the project being linted
3. The default ruleset

Creating a Ruleset

Any ruleset starts with the following base, shown in both JSON and YAML format:

{
  "$schema": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/schema.json",
  "version": 2,
  "axioms": {},
  "rules": {}
}

version: 2
axioms: {}
rules:

Where:

• $schema- points to the JSON schema for all Repolinter rulesets. This schema both validates the ruleset and makes the ruleset creation process a bit easier.
• version - specifies the ruleset version Repolinter should expect. Currently there are two versions: omitted for legacy config (example) and 2 for all others. Use 2 unless you know what you're doing.
• axiom - The axiom functionality, covered in Axioms.
• rules - The actual ruleset, covered in Rules.

Rules

Rules are objects of the following format:

"<rule-name>": {
  "level": "error" | "warning" | "off",
  "rule": {
    "type": "<rule-type>",
    "options": {
      // <rule-options>
    }
  },
  "where": ["condition=*"],
  "fix": {
    "type": "<fix-type>",
    "options": {
      // <fix-options>
    }
  },
  "policyInfo": "...",
  "policyUrl": "..."
}

<rule-name>:
  level: error | warning | off
  rule:
    type: <rule-type>
    options:
      <rule-options>
  where: [condition=*]
  fix:
    type: <fix-type>
    options:
      <fix-options>
  policyInfo: >-
    ...
  policyUrl: >-
    ...

• rule - The check to perform. Repolinter can perform any check listed under the rules documentation. Unlike eslint, Repolinter checks are designed to be reused and specialized: for example, the file-existence check can be used in a README-file-exists rule and a LICENSE-file-exists rule in the same ruleset. This allows a user to write a very specific ruleset from configuring generic checks.
• level - The error level to notify if the check fails. warning will not change the exit code and off will not run the check.
• where - Conditionally enable or disable this rule based off of axioms. Strings in this array follow the format of <axiom>=<value>, where value is either an axiom output or * to simply test if the axiom is enabled. If this option is present, this rule will only run if all specified axiom outputs are present. The available axioms in Repolinter can be found in the axioms documentation.
• fix (optional) - The action to perform if the check performed by rule fails. Repolinter can perform any action listed under fixes documentation.
• policyInfo, policyUrl (optional) - Information used by the formatter to indicate why the check exists from a policy perspective. Note: policyInfo will automatically have a period appended to it for formatting purposes.

A minimal example of a rule that checks for the existence of a README:

"readme-file-exists" : {
  "level": "error",
  "rule": {
    "type": "file-existence",
    "options": {
      "globsAny": ["README*"]
    }
  }
}

readme-file-exists:
  level: error
  rule:
    type: file-existence
    options:
      globsAny:
      - README*

Checking that the README matches a certain hash, and replacing it if not:

"readme-file-up-to-date" : {
  "level": "error",
  "rule": {
    "type": "file-hash",
    "options": {
      "globsAny": ["README*"],
      "algorithm": "sha256",
      "hash": "..."
    }
  },
  "fix": {
    "type": "file-create",
    "options": {
      "file": "README.md",
      "replace": true,
      "text": { "url": "www.example.com/mytext.txt" }
    }
  },
  "policyInfo": "Gotta keep that readme up to date",
  "policyUrl": "www.example.com/mycompany"
}

readme-file-up-to-date:
  level: error
  rule:
    type: file-hash
    options:
      globsAny:
      - README*
      algorithm: sha256
      hash: "..."
  fix:
    type: file-create
    options:
      file: README.md
      replace: true
      text:
        url: www.example.com/mytext.txt
  policyInfo: Gotta keep that readme up to date
  policyUrl: www.example.com/mycompany

Axioms

"axioms": {
  "<axiom-id>": "<axiom-target>"
}

axioms:
  <axiom-id>: axiom-target

Each axiom is configured as a key value pair in the axioms object, where <axiom-id> specifies the program to run and <axiom-target> specifies the target to be used in the where conditional. The available axiom IDs can be found in the axiom documentation. It should be noted that some axioms require external packages to run.

An example configuration using an axiom to detect the packaging system for a project:

{
  "$schema": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/schema.json",
  "version": 2,
  "axioms": {
    "packagers": "package-type"
  },
  "rules": {
    "this-only-runs-if-npm": {
      "level": "error",
      "where": ["package-type=npm"],
      "rule": { /* ... */ }
    }
  }
}

version: 2
axioms:
  packagers: package-type
rules:
  this-only-runs-if-npm:
    level: error
    where: [package-type=npm]
    rule:
      ...

Some axioms (ex. contributor-count) output numerical values instead of strings. For these axioms, numerical comparisons (<, >, <=, >=) can be also be specified in the where conditional. Note that if a numerical comparison is used for a non-numerical axiom, the comparison will always fail.

{
  "axioms": {
    "contributor-count": "contributors"
  },
  "rules": {
    "my-rule": {
      "where": ["contributors>6", "contributors<200"],
      // ...
    }
  }
}

axioms:
  contributor-count: contributors
rules:
  my-rule:
    where:
    - contributors>6
    - contributors<200
    rule:
      ...

Extending Rulesets

A ruleset can extend another ruleset, in which case the two files will be recursively merged. Extended rulesets can themselves extend additional rulesets up to 20 rulesets deep.

Extend a ruleset by including an "extends" top-level key which identifies a URL or file path:

{
  "extends": "https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/default.json"
  "rules": {
    # disable CI check
    "integrates-with-ci": {
      "level": "off"
    }
  }
}

extends: https://raw.githubusercontent.com/todogroup/repolinter/master/rulesets/default.json
rules:
  # disable CI check
  integrates-with-ci
    level: off
    ...

Relative paths are resolved relative to the location used to access the extending file. For example, if repolinter is invoked as:

repolinter -u http://example.com/custom-rules.yaml

And that ruleset includes extends: "./default.yaml", the path will be resolved relative to the original URL as http://example.com/default.yaml. If instead repolinter is invoked as:

repolinter -r /etc/repolinter/custom-rules.yaml

And that ruleset includes extends: "./default.yaml", the path will be resolved relative to the original file path as /etc/repolinter/default.yaml.

YAML and JSON rulesets can be extended from either format.

API

Repolinter also includes an extensible JavaScript API:

const repolinter = require('repolinter')
const result = await repolinter.lint('.')

This API allows the developer to have complete control over the configuration and formatting Repolinter should use. Documentation for this library can be found under API Documentation.

Going Further

• Rule Reference
• Fix Reference
• Axiom Reference
• API Reference
• Developer Guide

License

This project is licensed under the Apache 2.0 license.

Development

Development

Repolinter is broken down into 7 main components, each of which is contained in a subfolder:

• Axioms - located under axioms
• CLI - located under bin
• Rules - located under rules
• Fixes - located under fixes
• Formatters - located under formatters
• Utilities - located under lib
• Built-In Rulesets and Schema - located under rulesets

Axioms

Axioms are registered in axioms/axioms.js and in rulesets/schema.json. An axiom implementation consists of a module with the following interface:

async (fs: FileSystem) => Result

Where fs is a FileSystem scoped to the current repository.

The contents of the result should be an array of targets where t.path is a target that the axiom has determined is valid (ex. for language detecting axiom a possible result could be new Result('', [{ path: 'javascript' passed: true }, { path: 'typescript', passed: true}], true)). If the axiom fails to execute, it should return a failing result with an error message included instead of throwing an error.

Rules

A rule consists of two parts: a JavaScript module, which determines the rule's functionality and a JSON schema, which validates the rule's configuration options in rulesets. Rules are registered in rules/rules.js and rulesets/schema.json. Rules are also documented under rules.md.

Rule Configuration JSON Schema

The configuration JSON schema determines how rule.options should be validated for this rule. All JSON Schema tools supported by AJV all available, with a few important caveats:

• The file itself should always be named <rule-name>-config.json and be located under the rules folder.
• The top-level type should always be object.
• The $id field should be an absolute raw.githubusercontent.com URL to where the schema is hosted on GitHub (ex. https://raw.githubusercontent.com/todogroup/repolinter/master/rules/apache-notice-config.json). This allows IDEs such as VSCode to apply the schema via a URL.

To get started, you can use the following template:

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "$id": "https://raw.githubusercontent.com/todogroup/repolinter/master/rules/<rule-name>-config.json",
    "type": "object",
    "properties": {}
}

Due to limitations with JSON Schema, you also must register your rule configuration schema in rulesets/schema.json by adding the following item to the list under root.then.properties.rules.properties.rule.allOf:

{ "if": { "properties": { "type": { "const": "<rule-name>" } } }, "then": { "properties": { "options": { "$ref": "../rules/<rule-name>-config.json" } } } }

JavaScript Implementation

A rule implementation consists of a module with the following interface:

async (fs: FileSystem, opts /* type determined by your JSON schema */) => Result

Where fs is a FileSystem scoped to the current repository and opts is the options provided by the ruleset.

A rule implementation is encouraged to use Result#targets to show the individual files/patterns checked when processing the rule. Including filenames in Result#message is discouraged as it makes formatting difficult. If a rule fails to execute, it should throw an error.

Fixes

A fix, similar to a rule, consists of two parts: a JavaScript module, which determines the fix's functionality and a JSON schema, which validates the fix's configuration options in rulesets. Fixes are registered in fixes/fixes.js and rulesets/schema.json. Fixes are also documented under fixes.md.

Fix Configuration JSON Schema

Fix JSON schemas work identically to rule JSON schemas, with the only difference the respective names and paths.

To get started, you can use the following template:

{
    "$schema": "http://json-schema.org/draft-07/schema",
    "$id": "https://raw.githubusercontent.com/todogroup/repolinter/master/fixes/<fix-name>-config.json",
    "type": "object",
    "properties": {}
}

Similar as with rules, you must register the fix schema in rulesets/schema.json by adding the following item to the list under root.then.properties.rules.properties.fix.allOf (note that this is a different list than the rule registration list):

{ "if": { "properties": { "type": { "const": "<fix-name>" } } }, "then": { "properties": { "options": { "$ref": "../rules/<fix-name>-config.json" } } } }

JavaScript Implementation

Unlike rules, a fix implementation consists of a module with the following interface:

async (fs: FileSystem, options /* Type determined by your JSON schema */, targets: string[], dryRun: boolean) => Result

Where fs is a FileSystem scoped to the current repository, opts is the options provided by the ruleset, targets are filepaths which did not pass the rule associated with this fix, and dryRun determines if the fix is allowed to make changes to the repository.

The fix implementation is encouraged to use Result#targets to show the individual files/patterns changed. If the fix fails to execute, it should either return a failed result or throw an error.

Formatters

Formatters are exported by index.js and manually called by the CLI. A formatter implementation consists of the following interface:

interface Formatter {
  formatOutput(output: LintResult, dryRun: boolean): string
}

Formatters do not print to STDOUT instead choosing to return the output as a string.

If needed, a formatter can accept extra configuration from the ruleset through the formatOptions property, which will be directly passed through to LintResult#formatOptions. These options are not typed and are formatter dependent.

API Reference