Man rmlint

Good practice includes adding a $ anchor at the end of the regex.

Tips:

• l is useful for files like file.mp3 vs file.1.mp3 or file.mp3.bak.
• a can be used as last criteria to assert a defined order.
• o/O and h/H are only useful if there are hardlinks in the traversed path.
• o/O takes the number of hardlinks outside the traversed paths (and thereby minimizes/maximizes the overall number of hardlinks).

  Either -o or -O may be specified multiple times to get multiple outputs, including multiple outputs of the same format.

  Examples:

  $ rmlint -o json # Stream the JSON output to stdout $ rmlint -O csv:/tmp/rmlint.csv # Output an extra CSV file to /tmp
  -c--config=spec[=value] (default:none)

  Configure a format.

  This means, if the range is less than -q 0% to -Q 100%, than only partial duplicates are searched. We recommend the json formatter for every other scripting purpose.

  Available options:

  • omitfirst: Same as the -f / --omitfirst option in fdupes(1).

    WRONG ASSUMPTIONS ARE THE BIGGEST ENEMY OF YOUR DATA. The following hash families are available (in approximate descending order of cryptographic strength):

    sha3, blake,

    sha,

    highway, md

    metro, murmur, xxhash

    The weaker hash functions still offer excellent distribution properties, but are potentially more vulnerable to malicious crafting of duplicate files.

    The full list of hash functions (in decreasing order of checksum length) is:

    512-bit: blake2b, blake2bp, sha3-512, sha512

    384-bit: sha3-384,

    256-bit: blake2s, blake2sp, sha3-256, sha256, highway256, metro256, metrocrc256

    160-bit: sha1

    128-bit: md5, murmur, metro, metrocrc

    64-bit: highway64, xxhash.

    The use of 64-bit hash length for detecting duplicate files is not recommended, due to the probability of a random hash collision.

  -p--paranoid / -P--less-paranoid (default)

  Increase or decrease the paranoia of rmlint's duplicate algorithm.

  This speeds up subsequent runs on the same data set. You can use ls -i to show the inode number of a file and find -samefile <path> to find all hardlinks for a certain file.

• symlink: Tries to replace the duplicate file with a symbolic link to the original. several files may be chained together by the window.

  files that do not appear to have a duplicate. In this case you write the original directory after specifying a single //  on the command line.

  

  rmlint

  theharvesterA tool designed to be used in the early stages of a penetration test.gdm-stopImmediately stop running the GNOME Display Manager (GDM).poetry-buildBuild a Poetry package as a tarball and a wheel.npm-itThis command is an alias of `npm install-test`.git-subtreeMerge subtrees together or split repository into subtrees.pamsliceExtract one line of values out of a PAM image.lernapip3This command is an alias of `pip`.you-getDownload media contents (videos, audios, images) from the Web.jp2aConvert JPEG images to ASCII.bpythonA fancy interface to the Python interpreter.tlmgr-checkCheck the consistency of a TeX Live installation.brew-removeThis command is an alias of `brew uninstall`.waybarHighly customizable Wayland bar for Sway and Wlroots based compositors.pnmcolormapCreate quantization color map for a PNM image.docker-execThis command is an alias of `docker container exec`.d8convmvConvert filenames (NOT file content) from one encoding to another.apktoolReverse engineer APK files.xcvCut, copy, and paste in the command-line.xml-xmlnThis command is an alias of `xml pyx`.subfinderDiscover valid subdomains for websites.gopsList and diagnose Go processes currently running on your system.rawtoppmConvert a raw RGB stream to a PPM image.imapsyncEmail IMAP tool for syncing, copying, and migrating email mailboxes between two IMAP servers, one way, and without duplicates.pingSend ICMP ECHO_REQUEST packets to network hosts.ykinfoGet basic information from a YubiKey.aws-workmailManage Amazon WorkMail.git-stageThis command is an alias of `git add`.tslintA pluggable linting utility for TypeScript.nix-flake-showShow outputs provided by a flake.babeldRouting daemon for Babel which uses firewall-style filters.mycliA CLI for MySQL, MariaDB, and Percona that can do auto-completion and syntax highlighting.asciiquariumDisplay an animated aquarium in ASCII art inside the terminal.ncduDisk usage analyzer with an ncurses interface.crane-manifestGet the manifest of an image.indentChange the appearance of a C/C++ program by inserting or deleting whitespace.docker-container-statsDisplay a live stream of resource usage statistics for containers.pycodestyleCheck Python code against PEP 8 style conventions.kubectl-completionGenerate shell completion code for `kubectl` commands.kubectl-autoscaleCreate an autoscaler to intelligently scale pod count based on kubernetes cluster demands.rails-generateGenerate new Rails templates in an existing project.cpdfManipulate PDF files.bashBourne-Again SHell, an `sh`-compatible command-line interpreter.voltajs runtimes, npm and Yarn package managers, or any binaries from npm.wpacleanClean capture files to get only the 4-way handshake and a beacon.gcloud-kms-decryptDecrypt a ciphertext file using a Cloud KMS key.qownnotesMarkdown note-taking application.pamrgbatopngThis command has been superseded by `pamtopng`.ppmtoppmCopy a PPM image.resumeEasily setup a new resume.jj-rebaseMove revisions to different parent(s).git-show-treeShow a decorated tree graph with all branches of a Git repository, showing annotations.mmvMove and rename files in bulk.zmoreView `gzip` compressed files with `more`.svgtopamConvert an SVG file to Netpbm format.shcGeneric shell script compiler.scrcpyDisplay and control your Android device on a desktop.zTracks the most used (by frequency) directories and enables quickly navigating to them using string patterns or `regex`.docker-swarmA container orchestration tool.pulumi-stack-histThis command is an alias of `pulumi stack history`.crane-copyEfficiently copy a remote image from source to target while retaining the digest value.pbmtoiconThis command has been superseded by `pbmtosunicon`.gh-configChange configuration for GitHub CLI.ocamlcThe OCaml bytecode compiler.visudoSafely edit the sudoers file.gatsbyStatic site generator for React.ngrepFilter network traffic packets using `regex`.siegeHTTP loadtesting and benchmarking tool.cargoManage Rust projects and their module dependencies (crates).rustup-componentModify a toolchain's installed components.devpodLaunch reproducible development environments using Docker, Kubernetes, or SSH.batPrint and concatenate files.aws-sqsCreate, delete, and send messages to queues for the AWS SQS service.lighthouseAnalyzes web applications and web pages, collecting modern performance metrics and insights on developer best practices.sambapipe.pyExploit CVE-2017-7494 (SambaCry) to upload and load a shared object (SO) file on a vulnerable Samba server for remote code execution.git-squashSquash multiple commits into a single commit.tlmgr-platformManage TeX Live platforms.sfdk-deployDeploy build results to a device.cpCopy files and directories.waitress-servePure Python WSGI HTTP Server.git-magicAutomate add, commit, and push routines.bqA Python-based tool for BigQuery, Google Cloud's fully managed and completely serverless enterprise data warehouse.prowler-gcpAssess Google Cloud Platform (GCP) security best practices, audits, and compliance checks.bun-listThis command is an alias of `bun pm ls`.chrootRun command or interactive shell with special root directory.pbmtopgmConvert a PBM image to PGM by averaging areas surrounding individual pixels.blackfireMonitor, profile, and test a PHP application.pg_amcheckCheck for corruption in one or more PostgreSQL databases.jj-commitUpdate the description and create a new change on top.docker-rmThis command is an alias of `docker container rm`.git-get-tar-commit-idExtract commit ID from an archive created using `git archive`.cmatrixShows a scrolling Matrix like screen in the terminal.d2A modern diagram scripting language that turns text to diagrams.adb-installPush packages to a connected Android device or emulator.shfmtShell parser, formatter, and interpreter.leaveSet a reminder for when it's time to leave.iperf3Traffic generator for testing network bandwidth.cups-configShow technical information about your CUPS print server installation.apkleaksExpose URIs, endpoints, and secrets from APK files.duaDua (Disk Usage Analyzer): get the disk space usage of a directory.git-featureCreate or merge feature branches.pg_receivewalStream the write-ahead log from a running PostgreSQL cluster.gocryptfsEncrypted overlay filesystem written in Go.sha256sumCalculate SHA256 cryptographic checksums.flutterfireConfigure a Firebase project for your Flutter app.docker-rmiThis command is an alias of `docker image rm`.git-mktreeBuild a tree object using `ls-tree` formatted text.dateSet or display the system date.gettextTranslates a string using stored translations in a compiled `.mo` file.emacsThe extensible, customizable, self-documenting, real-time display editor.docker-psThis command is an alias of `docker container ls`.suiInteract with the Sui network.skopeoContainer image management toolbox.git-syncSync local branches with remote branches.pampickPick images out of a multi-image Netpbm stream.cargo-rustdocBuild the documentation of Rust packages.idevicesetlocationSimulate a location on an iOS device.typeormjs, browser, Cordova, Ionic, React Native, NativeScript, and Electron platforms.emacsclientOpen files in an existing Emacs server.jj-git-initCreate a new Git backed Jujutsu repo.cidrSimplifies IPv4/IPv6 CIDR network prefix management with counting, overlap checking, explanation, and subdivision.pyats-shellStart a pre-loaded pyATS interactive Python Shell to save time in prototyping.xml-elementsExtract elements and display the structure of an XML document.kaggle-configManage Kaggle configuration.git-create-branchCreate a Git branch in a repository.flyTool for concourse-ci.git-diff-treeCompares the content and mode of blobs found via two tree objects.git-obliterateDelete files and erase their history from a Git repository.phpenvA PHP version manager for development purposes.git-reflogShow a log of changes to local references like HEAD, branches, or tags.pnmtosirConvert a PNM file to a Solitaire Image Recorder file.dolt-mergeJoin two or more development histories together.git-bugreportCaptures debug information from the system and user, generating a text file to aid in the reporting of a bug in Git.wuzzInteractively inspect HTTP requests and responses.mp3infoViewer/editor for ID3v1 (but not ID3v2) tags of MP3 files.psyshA runtime developer console, interactive debugger, and REPL for PHP.dcflddEnhanced version of dd for forensics and security.7zFile archiver with a high compression ratio.licensorWrite licenses to `stdout`.npm-pingPing `npm` registry.az-storage-entityManage Azure Table storage entities.avrdudeDriver program for Atmel AVR microcontrollers programming.kcadm.shPerform administration tasks.ng-analyticsManage Angular CLI analytics settings.elasticsearch-nodeManage low-level Elasticsearch node operations such as shutdown, repurpose, or viewing info.conda-runRun an executable command in a conda environment.puppet-applyApply Puppet manifests locally.kubesealClient-side utility for encrypting Kubernetes secrets using the Bitnami Sealed Secrets controller.lprmCancel queued print jobs of a server.pip-configManage local and global configuration for pip.cargo-clippyA collection of lints to catch common mistakes and improve your Rust code.pipes.shA Bash script which draws randomly pathed pipes over the terminal.kshKorn Shell, a Bash-compatible command-line interpreter.composerA package-based dependency manager for PHP projects.eslinthub-forkFork a GitHub repo.

  The document is a list of dictionaries, where the first and last element is the header and the footer. If one criteria cannot decide between original and duplicate, the next one is tried.

  • m: keep lowest mtime (oldest)           M: keep highest mtime (newest)
  • a: keep first alphabetically            A: keep last alphabetically
  • p: keep first named path                P: keep last named path
  • d: keep path with lowest depth          D: keep path with highest depth
  • l: keep path with shortest basename     L: keep path with longest basename
  • r: keep paths matching regex            R: keep path not matching regex
  • x: keep basenames matching regex        X: keep basenames not matching regex
  • h: keep file with lowest hardlink count H: keep file with highest hardlink count
  • o: keep file with lowest number of hardlinks outside of the paths traversed by rmlint.
  • O: keep file with highest number of hardlinks outside of the paths traversed by rmlint.

  Alphabetical sort will only use the basename of the file and ignore its case.

  These files might have been modified and are silently ignored.

  By design, some options will not have any effect. If you had 10^73 different files, all the same size, then the chance of a false positive is still less than 1 in a billion. If you want to only find trees with the same hierarchy you should use --honour-dir-layout / -j.

  Output is deferred until all duplicates were found.

  rmlint uses the mtime to determine the modification timestamp if a checksum is outdated.