June 6, 2021

As usual, there is no single tool that does it all…

Wollmilchsau

ls: list directory contents

Good for listing directory contents and exploring properties of individual files.

ls

Fun flags (that can be combined):

  • ls --color=auto to turn on colors
  • ls -lh --time-style +'%Y-%m-%d %H:%M' to show a long listing format (-l) with human-readable file sizes (-h) and easy-to-read last modification times (--time-style)

tree: list contents of directories in a tree-like format

Good for exploring file hierarchies.

module load tree
tree

Fun flags (that can be combined):

  • tree -C to turn on colors
  • tree -sh to show size of each file (-s) with human-readable file sizes (-h)
  • tree -D --timefmt "%Y-%m-%d %H:%M" to show easy-to-read last modification times
  • tree -u to show username

Combine with less -r to make it scrollable:

tree -CshD | less -r

du: estimate file space usage

Good for checking the file sizes of directories.

du

Fun flags (that can be combined):

  • du -sh to display only a total for each argument in human-readable format
  • du -c to produce a grand total of all arguments

ncdu: NCurses Disk Usage

Good for checking the file sizes of directories interactively.

module load ncdu
ncdu

Fun flags (that can be combined):

  • ncdu --color dark to turn on colors
  • ncdu -e collect extended info (to show modification times)

Fun shortcuts:

  • m to show last modification times (only with -e)
  • d to delete selected file

Sadly, there is no way to show file ownership?

find: search for files in a directory hierarchy

The Swiss army knife of file search.

find

Get a list of files owned by myself:

find /mnt/research/quantgen/ -user $USER -type f

With a file size greater than 50 MiB:

-size +50M

Last modified more than one year ago:

-mtime +365

Printing last modification time and file size in bytes:

-printf '%p\t%AY-%Am-%Ad %TH:%TM\t%s\n'

rsync: a fast, versatile, remote (and local) file-copying tool

Good for getting data in and out of the HPCC.

Copying data from the HPCC:

rsync -avzP USERNAME@rsync.hpcc.msu.edu:/mnt/research/quantgen/projects/USERNAME/ DIR

Copy data to the HPCC:

rsync -avzP --chmod=Dg+s DIR/ USERNAME@rsync.hpcc.msu.edu:/mnt/research/quantgen/projects/USERNAME

Now it’s your turn