还在人工代码review？reviewdog和PMD了解一下

前言

代码reivew作为团队协同开发时确保代码质量的手段之一，在软件开发团队中非常常见。特别是对于刚入门不久的团队成员，通过代码review也可一定程度上避免一些低级错误，提升整个部分的代码健壮性。

一般来讲，通过代码review可发现的问题主要有以下几类：

1. 代码规范，如常量名应使用大写驼峰命名、一个方法代码量不超过300行、static变量应该使用final修饰……
2. 代码漏洞，如代码空指针、方法加锁后未主动释放锁、多线程下使用了线程不安全的集合……
3. 业务代码缺陷，如用户登录成功后应提示“登录成功”实际代码为“登录失败”、解析VRRP协议号时应为112代码写成了113、端口9200应识别为elasticsearch的默认端口识别成了mysql的默认端口……

对于上述中的代码规范问题与代码基础漏洞这类问题一般与业务无法，可由代码分析工具识别出来(一种可选的实现方案可使用antlr进行抽象语法树解析，此部分可参考笔者以前的文章：《使用antlr快速解析SQL》)。
诸如checkstyle、PMD、阿里巴巴规范p3c、findbugs、spotbugs这些常见的工具都是对这一类问题的实际运用。

而对于业务代码缺陷这类的问题，则往往需要对编码的具体业务有一定了解，此类问题的识别则需要具体的业务专家或业务测试用例来判别，如由共同开发同一模块的其他同学(结对编程)进行代码交叉review，或使用预定的业务单元测试进行用例覆盖等。

本文将介绍一种代码merge request提交后自动进行代码review的方案。通过既定的规则扫描，实现对增量代码的代码自动review与评论，为项目中的代码review提个速。

reviewdog介绍

如reviewdog的github(https://github.com/reviewdog/reviewdog)描述所述，reviewdog是一个由go语言开发的开源项目，它可实现对代码托管平台(如github、gitlab)项目中的代码进行自动提交评论，可与各种linter工具搭配使用，如golint、PMD、checkstyle等。

详细描述与官方示例可移步reviewdog管网查看，此处不再赘述。

对于历史项目，reviewdog支持增量检测，即：仅对某一个MR提交的增量部分进行评论。

以gitlab项目为例，它的一般工作流程大致为：

1. gitlab提交一个MR
2. MR通过gitlab CICD触发Lint tool进行代码静态分析
3. reviewdog读取步骤2静态分析的结果并进行解析，并使用git diff获取出当前MR增量的代码
4. 对静态分析结果与MR增量代码求交集，得到需要评论的代码，调用gitlab的api接口对代码提交评论

安装与运行

reviewdog的安装仅一个可执行文件，可在reviewdog的github中找到对应版本的release包。https://github.com/reviewdog/reviewdog/releases

以linux的x86系统、reviewdog0.20.1版本为例，在release中安装对应reviewdog文件为：reviewdog_0.20.1_Linux_x86_64.tar.gz

在linux中下载并授权：

#下载指定版本的reviewdog
wget https://github.com/reviewdog/reviewdog/releases/download/v0.20.1/reviewdog_0.20.1_Linux_x86_64.tar.gz
#解压
tar -zxvf reviewdog_0.20.1_Linux_x86_64.tar.gz
#拷贝到/usr/bin目录下
cp ./reviewdog /usr/bin/

验证一下：

reviewdog -version

再查看下支持的lint格式

reviewdog -list

root@ubuntu:~# reviewdog -list
rdjson		Reviewdog Diagnostic JSON Format (JSON of DiagnosticResult message)						- https://github.com/reviewdog/reviewdog
rdjsonl		Reviewdog Diagnostic JSONL Format (JSONL of Diagnostic message)							- https://github.com/reviewdog/reviewdog
diff		Unified Diff Format												- https://en.wikipedia.org/wiki/Diff#Unified_format
checkstyle	checkstyle XML format												- http://checkstyle.sourceforge.net/
sarif		SARIF JSON format												- https://sarifweb.azurewebsites.net/
ansible-lint	(ansible-lint -p playbook.yml) Checks playbooks for practices and behaviour that could potentially be improved	- https://github.com/ansible/ansible-lint
bandit		A tool designed to find common security issues in Python code.							- https://github.com/PyCQA/bandit.git
black		A uncompromising Python code formatter										- https://github.com/psf/black
brakeman	(brakeman --quiet --format tabs) A static analysis security vulnerability scanner for Ruby on Rails applications- https://github.com/presidentbeef/brakeman
buf		A new way of working with Protocol Buffers.									- https://github.com/bufbuild/buf
cargo-check	(cargo check -q --message-format=short) Check a local package and all of its dependencies for errors		- https://github.com/rust-lang/cargo
clippy		(cargo clippy -q --message-format=short) A bunch of lints to catch common mistakes and improve your Rust code	- https://github.com/rust-lang/rust-clippy
dotenv-linter	Lightning-fast linter for .env files. Written in Rust								- https://github.com/dotenv-linter/dotenv-linter
dotnet		(dotnet build -clp:NoSummary -p:GenerateFullPaths=true --no-incremental --nologo -v q) .NET Core CLI		- https://docs.microsoft.com/en-us/dotnet/core/tools/
erb-lint	(erblint --format compact) Lint your ERB or HTML files								- https://github.com/Shopify/erb-lint
eslint		(eslint [-f stylish]) A fully pluggable tool for identifying and reporting on patterns in JavaScript		- https://github.com/eslint/eslint
eslint-compact	(eslint -f compact) A fully pluggable tool for identifying and reporting on patterns in JavaScript		- https://github.com/eslint/eslint
fasterer	Speed improvements suggester											- https://github.com/DamirSvrtan/fasterer
flake8		Tool for python style guide enforcement										- https://flake8.pycqa.org/
go-consistent	Source code analyzer that helps you to make your Go programs more consistent					- https://github.com/quasilyte/go-consistent
golangci-lint	(golangci-lint run --out-format=line-number) GolangCI-Lint is a linters aggregator.				- https://github.com/golangci/golangci-lint
golint		linter for Go source code											- https://github.com/golang/lint
gosec		(gosec -fmt=golint) Golang Security Checker									- https://github.com/securego/gosec
govet		Vet examines Go source code and reports suspicious problems							- https://golang.org/cmd/vet/
haml-lint	Tool for writing clean and consistent HAML									- https://github.com/sds/haml-lint
hlint		Linter for Haskell source code											- https://github.com/ndmitchell/hlint
isort		A Python utility / library to sort Python imports								- https://github.com/PyCQA/isort
luacheck	(luacheck --formatter=plain) Lua linter and static analyzer							- https://github.com/luarocks/luacheck
misspell	Correct commonly misspelled English words in source files							- https://github.com/client9/misspell
msbuild		(msbuild /property:GenerateFullPaths=true /nologo /v:q) Microsoft Build Engine					- https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild
mypy		An optional static type checker for Python									- http://mypy-lang.org/
pep8		Python style guide checker											- https://pypi.python.org/pypi/pep8
phpstan		(phpstan --error-format=raw) PHP Static Analysis Tool - discover bugs in your code without running it!		- https://github.com/phpstan/phpstan
protolint	A pluggable linting utility for Protocol Buffer files								- https://github.com/yoheimuta/protolint
psalm		(psalm --output-format=text) Psalm is a static analysis tool for finding errors in PHP				- https://github.com/vimeo/psalm
puppet-lint	Check that your Puppet manifests conform to the style guide							- https://github.com/rodjek/puppet-lint
pydocstyle	A static analysis tool for checking compliance with Python docstring conventions				- https://github.com/PyCQA/pydocstyle
reek		(reek --single-line) Code smell detector for Ruby								- https://github.com/troessner/reek
remark-lint	Tool for writing clean and consistent markdown code								- https://github.com/remarkjs/remark-lint
rubocop		A Ruby static code analyzer, based on the community Ruby style guide						- https://github.com/rubocop-hq/rubocop
sbt		the interactive build tool											- http://www.scala-sbt.org/
sbt-scalastyle	Scalastyle - SBT plugin												- http://www.scalastyle.org/sbt.html
scalac		Scala compiler													- http://www.scala-lang.org/
scalastyle	Scalastyle - Command line											- http://www.scalastyle.org/command-line.html
slim-lint	Tool to help keep your Slim files clean and readable								- https://github.com/sds/slim-lint
sorbet		A fast, powerful type checker designed for Ruby									- https://github.com/sorbet/sorbet
standardjs	(standard) JavaScript style guide, linter, and formatter							- https://github.com/standard/standard
standardrb	(standard) Ruby style guide, linter, and formatter								- https://github.com/testdouble/standard
staticcheck	Golang Static Analysis												- https://staticcheck.io
stylelint	A mighty modern CSS linter											- https://github.com/stylelint/stylelint
tsc		TypeScript compiler												- https://www.typescriptlang.org/
tslint		An extensible linter for the TypeScript language								- https://github.com/palantir/tslint
typos		Source code spell checker											- https://github.com/crate-ci/typos
yamllint	(yamllint -f parsable) A linter for YAML files									- https://github.com/adrienverge/yamllint
root@ubuntu:~# 

SARIF介绍

目前reviewdog实际使用时与go项目整合的案例较多，go项目与reviewdog交互时使用的是golint格式。
在java方面使用reviewdog与java语言整合的案例资料较少，经过笔者的实践验证reviewdog和java项目整合也是可行的。需要用到的一个技术为PMD与reviewdog都支持的一种消息格式：SARIF。

SARIF的全称为：Static Analysis Results Interchange Format，即：静态分析结果交换格式。它是由结构化信息标准促进组织——OASIS(Organization for the Advancement of Structured Information Standards)提出的一种标准格式。

OASIS的官网为：https://www.oasis-open.org/

OASIS组织的成员也在每年持续增加中，部分成员截图如下：

关于SARIF的快速介绍可查看此文：《DevSecOps工具与平台交互的桥梁 – SARIF入门》

PMD介绍与安装

pmd，An extensible cross-language static code analyzer.
它是一个开源与编程语言无关的可扩展静态代码分析器。如在JAVA界比流行的阿里巴巴规范插件的实现也是基于PMD进行扩展开发的。

更多PMD的详细介绍与快速入门可移步它的介绍官网：https://pmd.github.io

本例为了演示，直接使用PMD的java-quickstart规则文件，地址为：https://github.com/pmd/pmd/blob/master/pmd-java/src/main/resources/rulesets/java/quickstart.xml

安装pmd也比较简单，它的发行包可直接在pmd的github中找到：
https://github.com/pmd/pmd/releases

以下载7.4.0版本为例，命令为：

wget https://github.com/pmd/pmd/releases/download/pmd_releases%2F7.4.0/pmd-dist-7.4.0-bin.zip
unzip pmd-dist-7.4.0-bin.zip

得到pmd的运行路径：/usr/local/pmd-bin-7.4.0/bin/pmd

pmd验证：

/usr/local/pmd-bin-7.4.0/bin/pmd -V

注意：pmd的运行依赖于java运行环境。jre安装命令示例：

apt install openjdk-17-jre-headless

gitlab与gitlab runner安装

在进行PMD和reviewdog联合演示前需要确保代码托管平台中的cicd插件已准备就绪。
这里以gitlab为例，对应的cicd插件则为gitlab runner。

如快速体验，gitlab可用docker快速启一个。示例：

docker run --name gitlab -p 9980:80 -p 9443:443 -p 9922:22 \--hostname 192.168.68.89 \--shm-size 2g\-v /gitlab/etc/gitlab:/etc/gitlab \-v /gitlab/var/log/gitlab:/var/log/gitlab \-v /gitlab/var/opt/gitlab:/var/opt/gitlab \-dit gitlab/gitlab-ce:latest

其中hostname填写为docker物理机的实际ip地址。

gitlab的默认密码可在映射的目录查看：

cat /gitlab/etc/gitlab/initial_root_password

然后登录gitlab，创建项目并为项目设置gitlab-runner。

然后根据示例安装好gitlab runner即可。

注册时填入的url为gitlab-server对应的ip和端口，token为此对应项目的token

sudo gitlab-runner register --url http://192.168.68.89:9980/ --registration-token GR13489414XKCJ3HPCuoz569yHC4v

需要注意的是gitlab-runner中的executor需要根据实际情况来填写，本例中为了演示方便使用的是shell，即在当前节点中运行runner

Enter an executor: shell, ssh, parallels, instance, custom, virtualbox, docker, docker-windows, docker+machine, kubernetes, docker-autoscaler:
shell

可使用gitlab-runner list命令验证是否注册成功

root@ubuntu:~# gitlab-runner list
Runtime platform                                    arch=amd64 os=linux pid=11348 revision=9882d9c7 version=17.2.1
Listing configured runners                          ConfigFile=/etc/gitlab-runner/config.toml
my-runner                                           Executor=shell Token=AU9QpietjBf6_zfx6t9Y URL=http://192.168.68.89:9980/

另外在gitlab的web界面中也可看到：

编写cicd脚本

在java项目源码根目录创建.gitlab-ci.yml文件，在内容如下：

variables:API_TOKEN: "glpat-CW77u3pMBdxnsnGLgwpp"
review-lint:script:- env- export REVIEWDOG_GITLAB_API_TOKEN=$API_TOKEN- export GITLAB_API=$CI_API_V4_URL- export CI_REPO_OWNER=$CI_PROJECT_NAMESPACE- export CI_REPO_NAME=$CI_PROJECT_NAME- export REVIEWDOG_INSECURE_SKIP_VERIFY=true- check_dirs=$(find ./ -type d -path '*/src/main/java')- /usr/local/pmd-bin-7.4.0/bin/pmd -V- /usr/local/pmd-bin-7.4.0/bin/pmd check -f sarif -R pmd/quickstart.xml -d $check_dirs --ignore-list pmd/ignore.txt > lint-report.out || true- reviewdog -f sarif -diff="git diff HEAD~ HEAD" -reporter=gitlab-mr-discussion -tee < lint-report.outonly:# 当前job只在执行MR提交时才执行- merge_requests

脚本中使用到的内置变量可参考gitlab文档：https://docs.gitlab.com/ee/ci/variables/predefined_variables.html

脚本中的REVIEWDOG_GITLAB_API_TOKEN为对应项目的Project Access Token，如没有则需要手动创建一个。

创建方式位于Settings的AccessTokens下。
如我这里的token为：glpat-CW77u3pMBdxnsnGLgwpp

验证

配置cicd流水线后，就可以提交一个mr进行验证了。
在mr中提交一个代码，让其不符合PMD中quickstart的规范：UnusedPrivateMethod

之后，在gitlab的MR中就会出现由reviewdog触发的自动评论了，示例截图如下：

如需自定义PMD规则，可参考阿里的P3C项目中的PMD规则实现。
https://github.com/alibaba/p3c/tree/master/p3c-pmd/src/main/resources/rulesets