Everyone is welcome here --- except those who have borrowed books from me for and have not returned them yet!

Linux tips and tricks

Posted on December 01, 2015 in howto

I always welcome suggestions to add new tips, correct or improve existing ones.

Unless otherwise indicated, the tips assume that one is using the bash shell.

Printing

To get a list of available printers:

lpstat -p -d

To check the status of all printers:

lpstat -a

To put the 'file.ps' in the printing queue of the printer 'printername':

lpr -P printername file.ps

To remove a printing job:

lprm job-id

(job-id is the number reported by the lpr or lpstat commands).

If your system uses the 'CUPS' printing system , you can control it by opening the following address in a browser:

http://localhost:631/

Find files or directories

To locate the file named 'filename' in all the subdirectories of the current directory:

find -iname 'filename'

You can use a pattern in place of 'filename', e.g. to return the list of all .doc files in the current directory and its subdirectories:

find -iname '*.doc'

You can specify that you only search for, e.g., directories, using the -type argument:

find -type d # list all subdirectorectries

You can filter on permissions

find -perm -o+x -ls -type f  # list all file with the execute flag on 'others'

You can find specify a time range:

find -ctime -10  # find the files created in the last 10 days:
find -mtime +30 -mtime -60  # find files modified in the last 30-60 days

You can also execute a command on each file:

find -name '*~' -exec rm '{}' ';'  # delete all files '*~' 
find -name '*.txt' -print0 | xargs -0 grep -l Alice # show files

Consult info find for more information.

To accelerate file search, you can generate a database of all filenames on you filesystem:

updatedb

And then use the command locate PATTERN. Note that the locate will return all files where PATTERN matches any substring in the full pathname (including directories).

Display the subdirectories as a tree

tree -d

Compare two files or two directories

To show all the lines that differ between file1 and file2:

diff file1 file2

To create a patch listing the changes from version1 to version2:

diff -aur version1 version2 >dir2.diff

To apply the patch to version1 and produce version2:

  patch -p1 <dir2.diff

For text files where the wrapping of paragraph may have changed, use 'wdiff'.

To compare two directories:

diff -r --brief dir1 dir2

Synchronize two directories

To copy the content of dir1 into dir2 without the copying the files that already exist and are the same, use rsync:

rsync -a dir1/ dir2

If you want to copy the newest files and delete the ones (make dir2 a mirror copy of dir1:)

rsync -a --delete dir1/ dir2

To synchronize two directories (in both directions), I use the program unison, which keeps two directories synchronized, transfering only new or modified files, in both directions:

unison localdir ssh://remotehost/remotedir

Check the amount of disk space occupied by a directory

du -h dirname

It may be useful to sort the subdirs by size:

du -k | sort -rn

To list files sorted by size:

ls -Sl

Check the space left on disks

To see the space available on the mounted partitions:

df -h

If there is a quota system that limits the amount of space you can use on your account, you can check how much:

quota

Get information about your system

To check how many cores are available on your machine

cat /proc/cpuinfo

How much physical memory

free -h

What is your IP number on the local network

ifconfig

To tell the version of the linux kernel

uname -a

Check the performance of your computer

You can see which processes are consuming CPU with top or htop:

htop

Check the available memory

free -h

You can follow memory usage in real time with

vmstat 10

If 'si' ou 'so' are high, your computer lacks memory and is using the swap (memory on disk).

You can check the IO volume and speed on the local drives

iostat -d 10

Check the speed of your ethernet connection. Three tools are available:

mii-tool

ethtool

iperf

Or the general network performance:

netstat -i 10

Large TX-ERR or RX-ERR indicate a problem.

Check which files are open and by which process

lsof

See http://www.thegeekstuff.com/2012/08/lsof-command-examples/

Adding a directory to the PATH

Most commands that you can execute from the command line are compiled programs or scripts. The shell (the program you are interacting with when typing commands on the terminal) searches for these programs in a list of directories. This list is stored in an environment variable called PATH. You can display its content:

echo $PATH

It is possible to find out which directory contains a given command by typing:

which command

(Note that when several directories contain a program with the same name, the first directory listed in the PATH has priority)

To add a new directory newdir to the list in PATH, type:

export PATH=newdir:$PATH

Add this line to the file '~/.bashrc': as the commands in this script are executed eevry time you open a new terminal, the PATH will be updated when you open a new terminal.

In case your SHELL is tsch or csh instead of bash, type:

set path = (newdir $path)
rehash

Place these lines in the file ~/.cshrc , to have the correct PATH the next time you open a shell.

Manipulating Images

Make sure to have ImageMagick installed (e.g. sudo apt install imagemagick on a Debian-based system)

To display an image (gif, .jpg, .png, .tiff, eps, ...)use:

display file.gif

To convert from one format to another:

convert file.jpg file.png

To resize an image:

convert img.png -resize 66%  img_small.png

To juxtapose several images:

montage -tile 4x4  *.png -geometry 1024x768 output.png

To superimpose images:

composite img1.png img2.png result.png

For more complex manipulations of bitmap image, I use The Gimp:

gimp file.jpg

To manipulate photographies, I like (Lightzone)[http://lightzoneproject.org/]

To edit vector graphics files, e.g. .svg, I use inkscape.

To draw graphs, I use dot.

Take a screenshot

To take a snapshot, that is, copy a portion of the screen into an image file, you can use ImageMagick's command import:

import file.png

You will then be able to select a rectanlg on the screen with the mouse, which will be copied in file.png.

Make a screencast

Voir http://www.linuxlinks.com/article/20090720142023520/Screencasting.html

To create a symbolic link (somewhat similar to a 'shortcut' in Windows):

ln -s filename newname

If you delete or move the file, the symbolic links will be 'dangling'.

To find and remove dangling links in a directory:

symlinks -rd directory

Convert a directory into a single file

Say you want to all the files in a directory in an email attachment. You can put a whole directory into a file ``file.tar'' with the command:

tar cf file.tar directory

Before sending it, it a good idea to compress ``file.tar'', as described in the next section.

Compress / uncompress files

To compressing a single file:

gzip file

To decompress it:

gunzip file.gz

To compress a directory:

tar czf aga.tar.gz directory # compressing
tar tzf aga.tar.gz # listing
tar xf aga.tar directory # uncompressing

.zip files:

zip -r archivename directories files
unzip -l archivename
unzip archivename

List all the programs currently running on the system

To list all the processes currently running:

ps -axlf

(you may omit the 'a' option if you want to list only the processes owned by you, and -l if you want less information)

The most important columns are 'time' and 'RSS' which show the time used by process since it started and the amount of real memory it takes.

If you want to list just some programs, for example ``matlab'', type

ps -ax | grep matlab

Another useful utility to list running processes is 'top':

top

The info is refreshed every second or so. Type 'M' to sort by memory usage, 'P4 to sort per CPU usage, and 'Q' to quit.

Under KDE, you can add the 'Ksysguard' applet to the Control Panel: this reports the CPU usage and amount of memory use.

Kill a program which no longer responds

It may happen that a program monopolizes most of the CPU, but does not longer respond to input. Such a program is crashed and should be destroyed.

For applications running in a terminal, first try 'Ctrl-C'.

If this does not work, or if the application is running in its own window but refusing to close, open a terminal and use 'ps' to locate the application (see section42) and note the number in the 'PID' column (``process identification number'').

Then, type:

kill PID

(in place of PID, use the number associated to the process listed in 'ps' output). Check if the program was destroyed with the 'ps' command; if not:

kill -9 PID

If the whole graphics system no longer responds, you can try to open a text mode terminal with Ctrl-Alt-F2 or Ctrl-Alt-F3, log in and kill the programs that cause problem (sometimes, it can be the 'X' program itself. 'X' (XWindow) is the program that manages the graphic display).

It the keyboard does not repond anymore, before switching off the computer, you can try to connect from another computer on the same network and kill the applications or do a proper shutdown (typing 'halt' on the command line).

Convert text files from DOS, Windows, Mac...

Texts files under unix, mac, dos and windows use differents codes for end-of-lines and accentuated characters. Linux typically uses the Latin-1 character encoding. The ``recode'' program converts from one format to the other (info recode).

To convert file.txt coming from a Macintosh:

recode mac..latin1 file.txt

To convert from DOS codepage 850 to unix latin1:

recode 850..latin1 file.txt

To just remove \^ M:

recode l1/crlf..l1 file.txt

To convert from Windows codepage 1250 to unix latin 1 (iso-8859-1):

recode ms-ee..l1 file.txt

See also iconv:

iconv -f ISO-8859-1 -t UTF-8 file.txt > "file-utf8.tex

Substitute strings in many files

perl -pi.bak -e 's/OLDSTING/NEWSTRING/g' FILELIST
# attention: to match words only, use s/\bOLDSTRING\b/NEWSTRING/g

Compute word frequencies in a text file

To compute the number of occurences of words in a file, say alice.txt:

tr -cs A-Za-z\' '\n' alice.txt | tr A-Z a-z | sort | uniq -c | sort -k1,1nr -k2 | head -10

    1637 the
    1083 '
    872 and
    730 to
    631 a
    540 she
    528 it
    513 of
    460 said
    410 i

Run a command on multiple files

It is sometime useful to run a command on many files. The shell bash provides 'for' loops:

for f in *.eps; do convert $f ${f%.eps}.jpg; done  # convert .eps files in the current directory into .jpg

Note that this is only valid if you are interacting with the shell 'Bash'. If you use another shell then you must type 'bash' first.

Count the number of files in a directory

To know how many files with extension .img are in the current directory:

ls *.img | wc -l

If you want to include all subdirectories:

find -type f -name '*.img' | wc -l

List files and subdirectories

ls
ls -1 # in one column
ls -l # with detailed format
ls -d # only directories
ls -l --sort=time | head  # most recent files only

Create subdirectories

mkdir -p subdirname

Copy, rename, move or delete files

To copy a file in the same directory, giving it name2:

cp file1 file2

To copy a file from the current directory to the existing directory dir:

cp file1 dir

To rename a file:

mv file1 file2

To move a file to the existing directory dir:

mv file1 dir

To delete a file:

rm file

To avoid being asked for confirmation:

 rm -f file

Copy, move or delete directories

To create a new directory:

mkdir newdir

To copy the directory 'dir' in the destination directory 'destdir':

cp -a dir destdir

(Note: the '-a' option does a recursive copy - that is includes the subdirectories - and preserves the attributes of files)

To move the whole directory 'dir' inside the existing 'destdir':

mv dir1 destdir

To rename directory 'dir' as 'dir2':

mv dir dir2

To delete the directory 'dir' and all its content:

rm -rf dir

Check or modify the rights of access to a file or a directory

When you use 'ls -l' to list the files in a directory, the first string of characters, made of 'x', 'r' 'w', '-'... specifies the access rights (Consult Understanding file permissions on Unix: a brief tutorial)

To allow everybody to read all the files in the current directory:

chmod a+r *

To avoid copying a file in several places on the same disk, it is a better idea to use a link:

ln existingname newname

Thus the same file can have several names (and be in several directories at the same time).

Who am I?

As far a the computer is concerned, the identity of the current user (that is its login), can be printed with:

whoami

Note that your login name, home directory, and shell are saved in the environment variables LOGNAME, HOME and shell:

echo $LOGNAME $HOME $SHELL

Change your identity

To temporally become newuser:

su - newuser

Of course, you will be prompted for newuser's password.

If you want to become root:

sudo -i

When you are done, type:

exit

Each login is associated to a UserID (UID), an integer, an to list of GroupIDs (GUID). You can list the information linked to the current login:

id

It is sometimes necessary to change your UID number, for example when you swap an external hardrive with a unix filesystem. Here is how to do it:

usermod -u <NEWUID> <LOGIN>
groupmod -g <NEWGID> <GROUP>
find / -user <OLDUID> -exec chown -h <NEWUID> {} \;
find / -group <OLDGID> -exec chgrp -h <NEWGID> {} \;
usermod -g <NEWGID> <LOGIN>

Change group

Check which groups you belong to using id, then use

newgrp  group

From now, the files and directories you create will belong to group group

To modify the group of already existing files in directory dir:

chgrp -R group dir

Which computer/system am I currently working on?

To display the network node name (also called the ``hostname''):

uname -a

The following also works:

hostname

Check who is logged on the computer

To see who is currently logged on the system, use

who

or

w

If you are superuser, you can see a journal of the logins with the command:

last

Connect to a remote computer

A secure method to connect to a remote computer:

ssh computername

or

ssh login@computer

Note that the remote computer must be running a 'sshd' server.

In the good old days, it was possible to connect with the commands:

rlogin computername
telnet computername

Note that the login and password were sent in clear over the network, and that is why ssh is prefered nowadays.

Execute commands on a remote computer, without login

ssh login@computername command

Setting up SSH

To avoid having to type your login password each time you use ssh or scp, you can setup SSH to use public and private keys to perform the authentification automagically.

First, you must generate keyfiles, once, on your local computer. To do so:

ssh-keygen

This generates, among other files, a public key stored in a file ~/.ssh/identity.pub). You now need to copy this key in the authorized_keys file inside the ~/.ssh directory of the remote computer you want to connect to.

ssh-copy-id  remotecomputer

If you have let an empty passphrase, you can know use ssh or scp without entering your password. But so can do anyone who access your account on your local computer.

So you may prefer to use a passphrase. To avoid having to type it each time you log to the remote computer, copy the following lines in your ~/.bash_profile:

eval `ssh-agent`
ssh-add < /dev/null

You will be prompted for the passphrase only once: when you login on the local computer (See the documentations of ssh-agent).

Display locally the interface of an XWindow program running on a remote computer

The X Window graphic system used by Linux allows to see on the local computer graphic windows generated by a program running on a remote computer.

On the local computer, type:

xhost +

On the remote computer, type:

setenv DISPLAY localname:0

(or 'export DISPLAY=localname:0' if the shell is 'bash')

Replace localname by the name of local computer. If you do not know it, type the following on the localcomputer:

uname -n

Keeping a remote session alive

On the remote computer, execute:

tmux

When you want to leave, press Ctrl-b d. The terminal is detached but not closed.

Next time you connect to this remote computer, to continue your work, you can access the session:

tmux a

See https://danielmiessler.com/study/tmux/ for a primer on tmux.

Copy files to or from a remote computer

scp -r misc remotelogin@computername:

rsync -avh localdir/ remotelogin@dest:path

tar  -cf - dir | ssh login@remotehost tar -xvf -

To synchronize bidirectionnaly:

unison

To 'aspire' web pages:

wget

For a better ftp

ncftp
lftp

Change the password

To change your password a local system:

passwd

To change your password on a network administrated with NIS (``yellowpages''):

yppasswd

Change the login shell

To change your password on the local system:

passwd

To change your login shell, e.g. from /bin/csh to /bin/bash:

ypchsh

or

chsh -s /bin/bash

Access files on a data CD or on a floppy

With some linux systems, you just insert the CD or the floppy and the content become available in the directory /mnt/cdrom or /mnt/floppy:

ls /mnt/cdrom
ls /mnt/floppy

If the floppy is not write-protected, you can create or copy files in /mnt/floppy just like in any ordinary folder.

Note that if you have several cdrom or floppy drives, they may have names cdrom1, cdrom2, floppy1,...

In some Linux systems, it is necessary to manually ``mount'' the cdrom or the floppy before accessing the files, and ``umount'' before ejecting it. For the cdrom:

mount /mnt/cdrom
ls /mnt/cdrom
...
umount /mnt/cdrom
eject

For the floppy:

mount /mnt/floppy
ls /mnt/floppy
umount /mnt/floppy

If you get an error message like ``mount: only root can do that'', ask the system administrator to grant you right to mount floppies by adding the ``user'' option the configuration file /etc/fstab. More information in the manual pages of mount and fstab:

man mount
man fstab

Concerning floppies, some systems have the mtools system (see 'man mtools') which provide the mdir and mcopy commands that emulate the old DOS commands dir and copy. It is not necessary to mount the floppy to use them.

Format a floppy

To format the floppy with an ext2 filesystem, and mount it:

fdformat /dev/fd0
mkfs -t ext2 /dev/fd0 
mount -t ext2 /dev/fd0 /mnt/floppy

This floppy can be read only on other linux systems. To be able to read it under Windows/DOS, you should use a DOS filesystem with mkdosfs in place of mkfs -t ext2:

mkdosfs /dev/fd0

Split a large file on several floppies

First compress the file, with gzip or bzip2 (see section41). If it still does not fit on a single floppy (1.4Mb), you can use the command split:

split -b1m file

This create a series of x?? files which you can copy on separate floppies.

To reassemble the files:

cat x* >file

Rip sound files from an audio CD

Maybe the easiest way to copy tracks from an audio cd it to open 'konqueror', and type 'audiocd:/' in the address bar. This will show you the content of the CD, which you can copy somewhere else. Copying from the mp3 or ogg folders will do the automatic translations for you.

Else, there are two programs with graphical interface which allow you to rip audio CD: 'grip' and 'kaudiocreator'.

On, the command line, you can use cdparanoia. For example, to rip an entire audio CD:

cdparanoia -B

To rip just one track:

cdparanoia -w track_number file.wav

Convert from wav to mp3

I use lame:

lame file.wav file.mp3

Convert from wav to ogg vorbis

I use oggenc:

oggenc file.wav -o file.ogg

Create a data CD

  1. Put all the files you want to save in a given directory, e.g. /tmp/mycd.
  2. Create an iso image:

    mkisofs -o cd.iso -J -R /tmp/mycd
    ls -l cd.iso
    

    Check that the resulting file cd.iso file is not too large to fit on the CD; if it less than 650Mb, this should be ok.

  3. Record on the cd (you must be root).

    You must know which is the device is associated to the CD writer drive.

    cdrecord -scanbus
    

    To determine the x,y,z scsi coordinates of your cd writer. If it does not appear listed, it may be because the ide-scsi parameter was no passed to the Linux kernel (See the HOWTO about CD Writing).

    To record, do:

    cdrecord dev=x,y,z -multi speed=0 -data cd.iso
    

Create an audio CD

To put on an audio CD all the *.wav files which are in the current directory:

cdrecord dev=x,y,z -pad speed=0 -audio *.wav

(x,y,z must be replaced by the numbers returned by cdrecord -scanbus)

Make backups

You can write backup scripts rsync but it has already been done many time. I recommend backintime

List the hosts in a NIS domain

If you are connected on a local network administrated by NIS (``yellow pages''), you can display the list of other computers on the network:

ypcat hosts

Which shell is running?

When you enter commands on the command line in a terminal, the text you type is interpreted by a program called the 'shell'. There are different shells that speak different dialects. To determine the shell you are communicating with, type:

echo $SHELL

Note: this does not work well for subshells:

bash
echo $SHELL
csh
echo $SHELL
exit
exit

Create a script to execute a series of commands

If you happen to often type the same series of commands, it is a good idea to create a script.

If it does not exist yet, create the directory \$HOME/bin.

Use a text editor to create a file 'myscript' in this directory, and type the series of commands.

The first line of the file should be:

#! /bin/bash

Save the file, then enter the command:

chmod +x ~/bin/myscript

You can now type ``myscript'' on the command line to execture the series of commands.

To go further, you should learn how to use arguments to scripts. There are tutorials on shell script programming on the web.

Get help. Find manuals

Many commands have associated ``man pages''. To read the man page associated, for example, to the command 'cp':

man cp

Some commands also have manuals in the form of 'info files':

info gawk

On many linux systems, there is additional documentation in the /usr/share/doc folder. The HOWTOs can be especially helpful.

Cut'n paste

Cutting & pasting under linux is not always straigtfoward. This is due to the fact that there are various systems of cut'n paste cohabitating.

To copy text, the following works with most applications:

  • Click the left button and drag the cursor over the text to be copied.
  • Click on the middle button to paste.

Note that this is very convenient: there no need to explicitly 'copy' the text.

If you use the window manager 'kde', there is a useful applet called 'klipper' located on the panel. Klipper keeps copies of the most recent clipboard contents. If a cut'n paste operation does not work, you may open klipper, select the relevant line, and retry to paste. It usually works.

If it does not work, then you can try the Cut/Copy/Paste functions from the applications' menus. Sometimes, it is necessary to save the region as a file in the first application, and insert this file in the second application.

Setup an ethernet card to access the internet

You need to know IP, MASK, GATEWAY, DNS, HOSTNAME and DOMAIN:

ifconfig eth0 IP netmask MASK up
route add -net default gw GATEWAY netmask 0.0.0.0 eth0
hostname HOSTNAME
echo ``domain DOMAIN'' >/etc/resolv.conf
echo "nameserver DNS" >>/etc/resolv.conf

Check the local disk IO speed

iostat -d 5

Dynamic libraries

To run, some programs need to access functions in dynamic libraries. Dynamic libraries have the extension .so. They are located in /lib, /usr/lib, /usr/local/lib...

To list the libraries needed by a program:

ldd program

After adding new a new dynamic library, e.g. in /usr/local/lib, you must run, as superuser:

ldconfig -n /usr/local/lib

It is possible, as a user, to tell linux to search libraries in a particular place, using the LD_LIBRARY_PATH variable. For more information about how dynamic libraries are accessed, consult the manual of ld.so:

man ld.so

Instal new software

If it come as a .tar.gz and contain a configure script

tar xzf package.tar.gz
cd package
./configure --prefix=$HOME & make & make install

This install the software in your home directory. To install it for every user, you need to omit the prefix option and be root when calling ``make install''.

If you are on a apt-based system (Debian, Ubuntu):

sudo apt-get install packagename

If you have the .deb file:

sudo dpkg -i file.deb

If you are on a rpm-based linux system, to install an rpm file:

rpm -i package.rpm

To check if the package is correctly installed:

rpm -V package

To remove it:

rpm -e package

Checking if a software package is installed

To check if, say, ghostscript is installed:

rpm -q ghostscript

You can get the list of all installed packages:

rpm -qa

Using a Palm Pilot

To backup data from the PalmPilot to the directory Palm:

export PILOTRATE=115200
export PILOTPORT=/dev/ttyS0
pilot-xfer --backup Palm

Common file types


typical extension file type application(s)

txt text or ascii file cat, less (view) vim, emacs (edit)

pdf Adobe PDF acroread, xpdf, evince, okular (view)

ps, eps postscript gv (view) pstops (rearrange) ps2pdf (convert)

html, htm web page links, konqueror, mozilla (view) soffice (create)

png, jpg, gif... graphic files display (view) import (snapshot) convert (convert) gimp (manipulate)

doc, xls, ppt Office document soffice

sxc, sxi, sxw OpenOffice document soffice

tex TeX and LaTeX documentz tex, latex, pdflatex (process)

dvi Dvi documents xdvi (view) dvips, dvipdf (convert to ps or pdf)

gz, Z Compressed file gunzip, zip, bunzip2, bzip2

tar tar archive tar tf (view) tar xf (extract) tar cf (create)

tar.gz compressed archive tar xzf (extract)

tar.bz2 Compressed tar archive tar xjf

zip zip archive unzip -l (view) unzip (extract) zip (create)


Linux tutorials, manuals... on the Net