Friday, 25 November 2016

Master Linux Command Line History

Master Linux Command Line History

When you are using Linux command line frequently, using the history effectively can be a major productivity boost. In fact, once you have mastered the 15 examples that I’ve provided here, you’ll find using command line more enjoyable and fun.

1. Display timestamp using HISTTIMEFORMAT

Typically when you type history from command line, it displays the command# and the command. For auditing purpose, it may be beneficial to display the timepstamp along with the command as shown below.
# export HISTTIMEFORMAT='%F %T '
# history | more
1  2008-08-05 19:02:39 service network restart
2  2008-08-05 19:02:39 exit
3  2008-08-05 19:02:39 id
4  2008-08-05 19:02:39 cat /etc/redhat-release

2. Search the history using Control+R

I strongly believe, this may be your most frequently used feature of history. When you’ve already executed a very long command, you can simply search history using a keyword and re-execute the same command without having to type it fully. Press Control+R and type the keyword. In the following example, I searched for red, which displayed the previous command “cat /etc/redhat-release” in the history that contained the word red.
# [Press Ctrl+R from the command prompt,
which will display the reverse-i-search prompt]
(reverse-i-search)`red': cat /etc/redhat-release
[Note: Press enter when you see your command,
which will execute the command from the history]
# cat /etc/redhat-release
Fedora release 9 (Sulphur)
Sometimes you want to edit a command from history before executing it. For e.g. you can search for httpd, which will display service httpd stop from the command history, select this command and change the stop to start and re-execute it again as shown below.
# [Press Ctrl+R from the command prompt,
which will display the reverse-i-search prompt]
(reverse-i-search)`httpd': service httpd stop
[Note: Press either left arrow or right arrow key when you see your
command, which will display the command for you to edit, before executing it]
# service httpd start

3. Repeat previous command quickly using 4 different methods

Sometime you may end up repeating the previous commands for various reasons. Following are the 4 different ways to repeat the last executed command.
  1. Use the up arrow to view the previous command and press enter to execute it.
  2. Type !! and press enter from the command line
  3. Type !-1 and press enter from the command line.
  4. Press Control+P will display the previous command, press enter to execute it

4. Execute a specific command from history

In the following example, If you want to repeat the command #4, you can do !4 as shown below.
# history | more
1  service network restart
2  exit
3  id
4  cat /etc/redhat-release

# !4
cat /etc/redhat-release
Fedora release 9 (Sulphur)

5. Execute previous command that starts with a specific word

Type ! followed by the starting few letters of the command that you would like to re-execute. In the following example, typing !ps and enter, executed the previous command starting with ps, which is ‘ps aux | grep yp’.
# !ps
ps aux | grep yp
root     16947  0.0  0.1  36516  1264 ?        Sl   13:10   0:00 ypbind
root     17503  0.0  0.0   4124   740 pts/0    S+   19:19   0:00 grep yp

6. Control the total number of lines in the history using HISTSIZE

Append the following two lines to the .bash_profile and relogin to the bash shell again to see the change. In this example, only 450 command will be stored in the bash history.
# vi ~/.bash_profile
HISTSIZE=450
HISTFILESIZE=450

7. Change the history file name using HISTFILE

By default, history is stored in ~/.bash_history file. Add the following line to the .bash_profile and relogin to the bash shell, to store the history command in .commandline_warrior file instead of .bash_history file. I’m yet to figure out a practical use for this. I can see this getting used when you want to track commands executed from different terminals using different history file name.
# vi ~/.bash_profile
HISTFILE=/root/.commandline_warrior
If you have a good reason to change the name of the history file, please share it with me, as I’m interested in finding out how you are using this feature.

8. Eliminate the continuous repeated entry from history using HISTCONTROL

In the following example pwd was typed three times, when you do history, you can see all the 3 continuous occurrences of it. To eliminate duplicates, set HISTCONTROL to ignoredups as shown below.
# pwd
# pwd
# pwd
# history | tail -4
44  pwd
45  pwd
46  pwd [Note that there are three pwd commands in history, after
executing pwd 3 times as shown above]
47  history | tail -4

# export HISTCONTROL=ignoredups
# pwd
# pwd
# pwd
# history | tail -3
56  export HISTCONTROL=ignoredups
57  pwd [Note that there is only one pwd command in the history, even after
executing pwd 3 times as shown above]
58  history | tail -4

9. Erase duplicates across the whole history using HISTCONTROL

The ignoredups shown above removes duplicates only if they are consecutive commands. To eliminate duplicates across the whole history, set the HISTCONTROL to erasedups as shown below.
# export HISTCONTROL=erasedups
# pwd
# service httpd stop
# history | tail -3
38  pwd
39  service httpd stop
40  history | tail -3

# ls -ltr
# service httpd stop
# history | tail -6
35  export HISTCONTROL=erasedups
36  pwd
37  history | tail -3
38  ls -ltr
39  service httpd stop
[Note that the previous service httpd stop after pwd got erased]
40  history | tail -6

10. Force history not to remember a particular command using HISTCONTROL

When you execute a command, you can instruct history to ignore the command by setting HISTCONTROL to ignorespace AND typing a space in front of the command as shown below. I can see lot of junior sysadmins getting excited about this, as they can hide a command from the history. It is good to understand how ignorespace works. But, as a best practice, don’t hide purposefully anything from history.
# export HISTCONTROL=ignorespace
# ls -ltr
# pwd
#  service httpd stop [Note that there is a space at the beginning of service,
to ignore this command from history]
# history | tail -3
67  ls -ltr
68  pwd
69  history | tail -3

11. Clear all the previous history using option -c

Sometime you may want to clear all the previous history, but want to keep the history moving forward.
# history -c

12. Subtitute words from history commands

When you are searching through history, you may want to execute a different command but use the same parameter from the command that you’ve just searched.
In the example below, the !!:$ next to the vi command gets the argument from the previous command to the current command.
# ls anaconda-ks.cfg
anaconda-ks.cfg
# vi !!:$
vi anaconda-ks.cfg
In the example below, the !^ next to the vi command gets the first argument from the previous command (i.e cp command) to the current command (i.e vi command).
# cp anaconda-ks.cfg anaconda-ks.cfg.bak
anaconda-ks.cfg
# vi  !^
vi anaconda-ks.cfg

13. Substitute a specific argument for a specific command.

In the example below, !cp:2 searches for the previous command in history that starts with cp and takes the second argument of cp and substitutes it for the ls -l command as shown below.
# cp ~/longname.txt /really/a/very/long/path/long-filename.txt
# ls -l !cp:2
ls -l /really/a/very/long/path/long-filename.txt
In the example below, !cp:$ searches for the previous command in history that starts with cp and takes the last argument (in this case, which is also the second argument as shown above) of cp and substitutes it for the ls -l command as shown below.
# ls -l !cp:$
ls -l /really/a/very/long/path/long-filename.txt

14. Disable the usage of history using HISTSIZE

If you want to disable history all together and don’t want bash shell to remember the commands you’ve typed, set the HISTSIZE to 0 as shown below.
# export HISTSIZE=0
# history
# [Note that history did not display anything]

15. Ignore specific commands from the history using HISTIGNORE

Sometimes you may not want to clutter your history with basic commands such as pwd and ls. Use HISTIGNORE to specify all the commands that you want to ignore from the history. Please note that adding ls to the HISTIGNORE ignores only ls and not ls -l. So, you have to provide the exact command that you would like to ignore from the history.
# export HISTIGNORE="pwd:ls:ls -ltr:"
# pwd
# ls
# ls -ltr
# service httpd stop

# history | tail -3
79  export HISTIGNORE="pwd:ls:ls -ltr:"
80  service httpd stop
81  history
[Note that history did not record pwd, ls and ls -ltr]


Top 5 Best Linux OS Distributions

Based on this data, the top spot in the best Linux distribution list goes to…

Ubuntu

My personal favorite was Ubuntu for desktop (#1 in this list) and Red Hat for servers (#5 in this list).

If you are new to any of the distros listed in the top 5, read the rest of the article to understand little bit more about those distros and find out whether your favorite Linux distribution made it in the top 5.

Linux Distro Review
Fig: Favorite Linux Distribution Voting Results

1. Ubuntu

Ubuntu 8.04 LTS Desktop
Like most of you, Ubuntu is my #1 choice for desktop Linux. I use it both at home and work. Ubuntu is the #1 in the Linux desktop market and some use Ubuntu for the servers also. Ubuntu offers the following three editions.
  • Ubuntu Desktop Edition
  • Ubuntu Server Edition
  • Ubuntu Notebook Remix

Additional Details:

Refer to our Ubuntu Tips and Tricks article series.

2. Debian

Debian 4.0r8, or etch
Debian is also called as Debian GNU/Linux, as most of the basic OS tools comes from the GNU Project. Lot of other famous distributions are based on Debian, which includes our #1 distro Ubuntu and many others — such as Knoppix, Linspire, Damn Small Linux etc.,

Additional Details:

Read more about Debian Distribution at wikipedia.

3. Fedora

Fedora 10 Server Edition
Fedora is sponsored by Red Hat. If you are interested in experimenting with the the leading technologies, you should use fedora, as the release cycle is very short and fedora tends to include the latest technology software/packages in it’s distribution.

Additional Details:

Read more about Fedora Distribution at wikipedia.

4. CentOS

CentOS 5 Linux Distro
If your organization does not want to spend money on purchasing Red Hat support, but still want all the benefits of the red-hat distribution, this is obviously the best choice, as this is totally based on the red-hat enterprise Linux.

As you can imagine the Nort American Enterprise Linux vendor mentioned in the quote below is Red Hat.
From the CentOS website: CentOS 2, 3, and 4 are built from publically available open source SRPMS provided by a prominent North American Enterprise Linux vendor. CentOS is designed for people who need an enterprise class OS without the cost or support of the prominent North American Enterprise Linux vendor.

Additional Details:

Read more about CentOS Distribution at wikipedia

5. Red Hat

Linux Red Hat 4 Enterprise Linux for Server
This is my favorite server distribution.  If an organization doesn’t mind spending dollars on purchasing the red-hat support, this is always my #1 recommendation to any organization who runs mission critical applications.

On a side note, one of the reason I like Red Hat Linux for mission critical production application is that Red Hat tends to take some of the new features from Fedora, which is well tested by the community.

Additional Details:

Read more about Red Hat Distribution at wikipedia.

Awesome Linux Articles

Following are few awesome 15 examples articles that you might find helpful.
Note: To get high quality free Linux articles, subscribe to the geek stuff.

UNIX / Linux: 10 Netstat Command Examples

Netstat command displays various network related information such as network connections, routing tables, interface statistics, masquerade connections, multicast memberships etc.,
In this article, let us review 10 practical unix netstat command examples.

1. List All Ports (both listening and non listening ports)

List all ports using netstat -a

# netstat -a | more
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
udp        0      0 *:bootpc                *:*                                

Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6135     /tmp/.X11-unix/X0
unix  2      [ ACC ]     STREAM     LISTENING     5140     /var/run/acpid.socket

List all tcp ports using netstat -at

# netstat -at
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
tcp        0      0 localhost:ipp           *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN

List all udp ports using netstat -au

# netstat -au
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 *:bootpc                *:*
udp        0      0 *:49119                 *:*
udp        0      0 *:mdns                  *:*

2. List Sockets which are in Listening State

List only listening ports using netstat -l

# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:ipp           *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN
udp        0      0 *:49119                 *:*

List only listening TCP Ports using netstat -lt

# netstat -lt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 localhost:30037         *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp6       0      0 localhost:ipp           [::]:*                  LISTEN

List only listening UDP Ports using netstat -lu

# netstat -lu
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
udp        0      0 *:49119                 *:*
udp        0      0 *:mdns                  *:*

List only the listening UNIX Ports using netstat -lx

# netstat -lx
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     6294     private/maildrop
unix  2      [ ACC ]     STREAM     LISTENING     6203     public/cleanup
unix  2      [ ACC ]     STREAM     LISTENING     6302     private/ifmail
unix  2      [ ACC ]     STREAM     LISTENING     6306     private/bsmtp

3. Show the statistics for each protocol

Show statistics for all ports using netstat -s

# netstat -s
Ip:
    11150 total packets received
    1 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    11149 incoming packets delivered
    11635 requests sent out
Icmp:
    0 ICMP messages received
    0 input ICMP message failed.
Tcp:
    582 active connections openings
    2 failed connection attempts
    25 connection resets received
Udp:
    1183 packets received
    4 packets to unknown port received.
.....

Show statistics for TCP (or) UDP ports using netstat -st (or) -su

# netstat -st

# netstat -su

4. Display PID and program names in netstat output using netstat -p

netstat -p option can be combined with any other netstat option. This will add the “PID/Program Name” to the netstat output. This is very useful while debugging to identify which program is running on a particular port.
# netstat -pt
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        1      0 ramesh-laptop.loc:47212 192.168.185.75:www        CLOSE_WAIT  2109/firefox
tcp        0      0 ramesh-laptop.loc:52750 lax:www ESTABLISHED 2109/firefox

5. Don’t resolve host, port and user name in netstat output

When you don’t want the name of the host, port or user to be displayed, use netstat -n option. This will display in numbers, instead of resolving the host name, port name, user name.
This also speeds up the output, as netstat is not performing any look-up.
# netstat -an
If you don’t want only any one of those three items ( ports, or hosts, or users ) to be resolved, use following commands.
# netsat -a --numeric-ports

# netsat -a --numeric-hosts

# netsat -a --numeric-users

6. Print netstat information continuously

netstat will print information continuously every few seconds.
# netstat -c
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 ramesh-laptop.loc:36130 101-101-181-225.ama:www ESTABLISHED
tcp        1      1 ramesh-laptop.loc:52564 101.11.169.230:www      CLOSING
tcp        0      0 ramesh-laptop.loc:43758 server-101-101-43-2:www ESTABLISHED
tcp        1      1 ramesh-laptop.loc:42367 101.101.34.101:www      CLOSING
^C

7. Find the non supportive Address families in your system

netstat --verbose
At the end, you will have something like this.
 netstat: no support for `AF IPX' on this system.
 netstat: no support for `AF AX25' on this system.
 netstat: no support for `AF X25' on this system.
 netstat: no support for `AF NETROM' on this system.

8. Display the kernel routing information using netstat -r

# netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.1.0     *               255.255.255.0   U         0 0          0 eth2
link-local      *               255.255.0.0     U         0 0          0 eth2
default         192.168.1.1     0.0.0.0         UG        0 0          0 eth2
Note: Use netstat -rn to display routes in numeric format without resolving for host-names.

9. Find out on which port a program is running

# netstat -ap | grep ssh
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        1      0 dev-db:ssh           101.174.100.22:39213        CLOSE_WAIT  -
tcp        1      0 dev-db:ssh           101.174.100.22:57643        CLOSE_WAIT  -
Find out which process is using a particular port:
# netstat -an | grep ':80'

10. Show the list of network interfaces

# netstat -i
Kernel Interface table
Iface   MTU Met   RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0       1500 0         0      0      0 0             0      0      0      0 BMU
eth2       1500 0     26196      0      0 0         26883      6      0      0 BMRU
lo        16436 0         4      0      0 0             4      0      0      0 LRU
Display extended information on the interfaces (similar to ifconfig) using netstat -ie:
# netstat -ie
Kernel Interface table
eth0      Link encap:Ethernet  HWaddr 00:10:40:11:11:11
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:f6ae0000-f6b00000

How to Setup Git Repository and Credentials for Jenkins Jobs

 Jenkins is an open source automation server, which will help you to automate the application deployment in your infrastructure.
From Jenkins, for deployment, you can connect to any kind of source code control system, and pull the source, build it, and deploy it automatically to one or more servers.
In this tutorial, we’ll explain how you can create a new Jenkins Job, and connect it to a Git repository to pull the source code for building.

Install Jenkins Git Plugin

For this, you should have “Git Plugin” installed in your Jenkins.
Login to Jenkins UI using your admin account, and go to “Manage Jenkins” -> Manage Plugins -> Click on “Installed” Tab -> From here, search for “git plugin” in the filter.
If the “Git plugin” is already installed, it will display it here as shown below.
Jenkins Git Plugins List

During Jenkins installation, if you’ve chosen to install the “recommended plugin”, you’ll already have “git plugin” as shown above.
If you don’t see it, then install it. Go to “Available” tab, search for “git plugin”, and click on “Install without restart”.
From your plugins list, if you see that “Git Plugin” is an older version, then upgrade it to the latest version as explained here: How to Upgrade Jenkins to New Version and Update Plugins

Create New Jenkins Job

Click on “New” to create a new jenkins job. Select “Free Style Project” as type as shown below. Name: Dev App Build. Type: Free Style Project
Jenkins Git Create New Job
By default this will go into editing the job after creating it. In the edit mode, you’ll see multiple tabs on the top, and “General” tab will be selected.

Select “Git” for Source Code Management

For this particular job, click on “Source Code Management” tab. This will have the following options:
  • None (This option will be selected by default).
  • CVS
  • CVS Projectset
  • Git
  • Subversion
Click on “Git” to select it. This will expand the section, and show all git related parameters as shown below.
Jenkins Job Git Source Code Management Tab
Under the Git section, in the Repositories section, you’ll see the following:
Repository URL: This will be the URL of your remote repository. The syntax is same as what you’ll specify in the git command line. For example: username@git-server-ip:/git/repository/full/path
The following are some possible syntax for the repository URL:
  • ssh://git@github.com/github/git.git
  • git@github.com:github/git.git
  • ssh://user@other.host.com/~/repos/R.git
  • https://github.com/github/git.git
  • git://github.com/github/git.git
Credentials: By default, this will be “None”. There will an “Add” button next to it. From here, you can specify the credentials that are required for the Git URL specified above.
Advanced button: When you click on the “Advanced” button, you’ll see two more text boxes. For most cases, you don’t need to enter anything here.
  • Name – You can leave this empty, and Jenkins will automatically generate unique name. This “name” is exactly the same as the one used in the git remote command.
  • Refspec – The default refspec is “+refs/heads/*:refs/remotes/REPOSITORYNAME/*”. This is Ok for most of you, and you can leave this empty. By default will be retrieve all the branch heads as remotes/repositoryname/branchname. Instead, if you always want to retrive just one branch, you may want to modify this.
Add repository button: Click on “Add repository” to create this as the default git repository to be used by this particular Jenkins Job.
Branches to build section: In this section under “Branch Specifier (blank for ‘any’):”, the default value is */master. For most cases, you can leave this to default. This will track only the master branch. But, if you leave this blank, all branches will be examined for changes and built. The syntax for this field is: refs/heads/
The following are various possible examples for the “Branch Specifier” field:
  • refs/heads//origin/master
  • remotes/origin/master
  • refs/remotes/origin/master
  • tag-name
  • refs/tags/tag-name
  • commit-id
  • ${ENV_VARIABLE}
  • ${TREEISH}
  • refs/tags/${TAGNAME},…
  • Wildcards: e,g: *, **
  • :regular-expression
Add Branch button: Click on this button to add the branch specified in the above “Branch Specifier” field to be used.
Repository browser: By default, this drop down list will be set to “Auto”. In most cases, you can just leave it to default. This will create the links for the “changes” view inside the Jenkins to an external system for browsing. The “Auto” option will automatically choose the best possible repository browser for the selected SCM.
The following are possible values for this Repository browser:
  • Auto
  • AssemblaWeb
  • FishEye
  • Kiln
  • Microsoft Team Foundation Server/Visual Studio Team Services
  • bitbucketweb
  • cgit
  • gitblit
  • githubweb
  • gitiles
  • gitlab
  • gitlist
  • gitoriousweb
  • gitweb
  • phabricator
  • redmineweb
  • rhodecode
  • stash
  • viewgit
When you create a new Jenkins job, make sure the security is enabled on that particular job as explained here: How to Setup User Security on Jenkins with Project Matrix Authorization

Setup Jenkins Credentials for Git

When you create the repository, you might get the following error:
Repository URL error 1: ramesh@192.168.101.2:/home/git/devapp
Failed to connect to repository : Command "git ls-remote -h ramesh@192.168.101.2:/home/git/devapp HEAD" returned status code 128:
stdout:
stderr: Host key verification failed.
fatal: The remote end hung up unexpectedly
Typically, you’ll get the above error message, when your “Credentials” under “Git” is set to “None”.
To add a credential, click on “Add” next to “Credentials” -> Select “Jenkins Credential Provider”, this will display the following add credentials screen.
Jenkins Git Create Credentials
  • Domain: By default “Global credentials (unrestricted)” is selected. Other option is: “Username and password”. Use default.
  • Kind: The following are the possible options:
    • Username and password (This is shown in the above screenshot).
    • SSH Username with private key
    • Secret file
    • Secret text
    • Certificate
  • Username – Specify the username that will be used to connect to the remote git repository
  • Password – Specify the password for the above username.
  • ID – Leave this empty. This is an internal unique ID that are used to identify this credentials in the job. When you leave this empty, Jenkins will auto generate an id.
  • Description – Descriptive notes about this credentials
If you choose, the SSH username with private key, you’ll see the following screen:
Jenkins Git Create Credentials SSH
  • Kind: SSH Username with Private Key
  • Username – Specify the username that will be used to connect to the remote git repository
  • Private Key: The following are three options for this:
    • Enter directly
    • From a file on Jenkins master
    • From the jenkins master ~/.ssh
  • Passphrase – The passphrase for your key
  • ID – The unique ID (same as what is explained above)
  • Description – Notes
Once the credentials are created, complete the remaining sections in the “Git” section under the “Source Code Management” tab for your particular job.
After this, if you want to view all the credentials that you’ve created so far in Jenkins, go to “Jenkins” -> Click on “Credentials” on the menu, which will list all the credentials that you’ve created inside Jenkins so far as shown below.
Jenkins Git Credentials List
So far we’ve created a new Jenkins job and associated it to a specific git repository, and specified which branch from the repository to be used when this particular job is executed.

grep command examples .

2. grep command examples

Search for a given string in a file (case in-sensitive search).
$ grep -i "the" demo_file
Print the matched line, along with the 3 lines after it.
$ grep -A 3 -i "example" demo_text
Search for a given string in all files recursively
$ grep -r "ramesh" *

3. find command examples

Find files using file-name ( case in-sensitve find)
# find -iname "MyCProgram.c"
Execute commands on files found by the find command
$ find -iname "MyCProgram.c" -exec md5sum {} \;
Find all empty files in home directory
# find ~ -empty

4. ssh command examples

Login to remote host
ssh -l jsmith remotehost.example.com
Debug ssh client
ssh -v -l jsmith remotehost.example.com
Display ssh client version
$ ssh -V
OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003

5. sed command examples

When you copy a DOS file to Unix, you could find \r\n in the end of each line. This example converts the DOS file format to Unix file format using sed command.
$sed 's/.$//' filename
Print file content in reverse order
$ sed -n '1!G;h;$p' thegeekstuff.txt
Add line number for all non-empty-lines in a file
$ sed '/./=' thegeekstuff.txt | sed 'N; s/\n/ /'

6. awk command examples

Remove duplicate lines using awk
$ awk '!($0 in array) { array[$0]; print }' temp
Print all lines from /etc/passwd that has the same uid and gid
$awk -F ':' '$3==$4' passwd.txt
Print only specific field from a file.
$ awk '{print $2,$5;}' employee.txt

7. vim command examples

Go to the 143rd line of file
$ vim +143 filename.txt
Go to the first match of the specified
$ vim +/search-term filename.txt
Open the file in read only mode.
$ vim -R /etc/passwd

8. diff command examples

Ignore white space while comparing.
# diff -w name_list.txt name_list_new.txt

2c2,3
< John Doe --- > John M Doe
> Jason Bourne

9. sort command examples

Sort a file in ascending order
$ sort names.txt
Sort a file in descending order
$ sort -r names.txt
Sort passwd file by 3rd field.
$ sort -t: -k 3n /etc/passwd | more

10. export command examples

To view oracle related environment variables.
$ export | grep ORACLE
declare -x ORACLE_BASE="/u01/app/oracle"
declare -x ORACLE_HOME="/u01/app/oracle/product/10.2.0"
declare -x ORACLE_SID="med"
declare -x ORACLE_TERM="xterm"
To export an environment variable:
$ export ORACLE_HOME=/u01/app/oracle/product/10.2.0

11. xargs command examples

Copy all images to external hard-drive
# ls *.jpg | xargs -n1 -i cp {} /external-hard-drive/directory
Search all jpg images in the system and archive it.
# find / -name *.jpg -type f -print | xargs tar -cvzf images.tar.gz
Download all the URLs mentioned in the url-list.txt file
# cat url-list.txt | xargs wget –c

12. ls command examples

Display filesize in human readable format (e.g. KB, MB etc.,)
$ ls -lh
-rw-r----- 1 ramesh team-dev 8.9M Jun 12 15:27 arch-linux.txt.gz
Order Files Based on Last Modified Time (In Reverse Order) Using ls -ltr
$ ls -ltr
Visual Classification of Files With Special Characters Using ls -F
$ ls -F

13. pwd command

pwd is Print working directory. What else can be said about the good old pwd who has been printing the current directory name for ages.

14. cd command examples

Use “cd -” to toggle between the last two directories
Use “shopt -s cdspell” to automatically correct mistyped directory names on cd

15. gzip command examples

To create a *.gz compressed file:
$ gzip test.txt
To uncompress a *.gz file:
$ gzip -d test.txt.gz
Display compression ratio of the compressed file using gzip -l
$ gzip -l *.gz
         compressed        uncompressed  ratio uncompressed_name
              23709               97975  75.8% asp-patch-rpms.txt

16. bzip2 command examples

To create a *.bz2 compressed file:
$ bzip2 test.txt
To uncompress a *.bz2 file:
bzip2 -d test.txt.bz2

17. unzip command examples

To extract a *.zip compressed file:
$ unzip test.zip
View the contents of *.zip file (Without unzipping it):
$ unzip -l jasper.zip
Archive:  jasper.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
    40995  11-30-98 23:50   META-INF/MANIFEST.MF
    32169  08-25-98 21:07   classes_
    15964  08-25-98 21:07   classes_names
    10542  08-25-98 21:07   classes_ncomp

18. shutdown command examples

Shutdown the system and turn the power off immediately.
# shutdown -h now
Shutdown the system after 10 minutes.
# shutdown -h +10
Reboot the system using shutdown command.
# shutdown -r now
Force the filesystem check during reboot.
# shutdown -Fr now

19. ftp command examples

Both ftp and secure ftp (sftp) has similar commands. To connect to a remote server and download multiple files, do the following.
$ ftp IP/hostname
ftp> mget *.html
To view the file names located on the remote server before downloading, mls ftp command as shown below.
ftp> mls *.html -
/ftptest/features.html
/ftptest/index.html
/ftptest/othertools.html
/ftptest/samplereport.html
/ftptest/usage.html

20. crontab command examples

View crontab entry for a specific user
# crontab -u john -l
Schedule a cron job every 10 minutes.
*/10 * * * * /home/ramesh/check-disk-space

21. service command examples

Service command is used to run the system V init scripts. i.e Instead of calling the scripts located in the /etc/init.d/ directory with their full path, you can use the service command.
Check the status of a service:
# service ssh status
Check the status of all the services.
service --status-all
Restart a service.
# service ssh restart

22. ps command examples

ps command is used to display information about the processes that are running in the system.
While there are lot of arguments that could be passed to a ps command, following are some of the common ones.
To view current running processes.
$ ps -ef | more
To view current running processes in a tree structure. H option stands for process hierarchy.
$ ps -efH | more

23. free command examples

This command is used to display the free, used, swap memory available in the system.
Typical free command output. The output is displayed in bytes.
$ free
             total       used       free     shared    buffers     cached
Mem:       3566408    1580220    1986188          0     203988     902960
-/+ buffers/cache:     473272    3093136
Swap:      4000176          0    4000176
If you want to quickly check how many GB of RAM your system has use the -g option. -b option displays in bytes, -k in kilo bytes, -m in mega bytes.
$ free -g
             total       used       free     shared    buffers     cached
Mem:             3          1          1          0          0          0
-/+ buffers/cache:          0          2
Swap:            3          0          3
If you want to see a total memory ( including the swap), use the -t switch, which will display a total line as shown below.
ramesh@ramesh-laptop:~$ free -t
             total       used       free     shared    buffers     cached
Mem:       3566408    1592148    1974260          0     204260     912556
-/+ buffers/cache:     475332    3091076
Swap:      4000176          0    4000176
Total:     7566584    1592148    5974436

24. top command examples

top command displays the top processes in the system ( by default sorted by cpu usage ). To sort top output by any column, Press O (upper-case O) , which will display all the possible columns that you can sort by as shown below.
Current Sort Field:  P  for window 1:Def
Select sort field via field letter, type any other key to return

  a: PID        = Process Id              v: nDRT       = Dirty Pages count
  d: UID        = User Id                 y: WCHAN      = Sleeping in Function
  e: USER       = User Name               z: Flags      = Task Flags
  ........
To displays only the processes that belong to a particular user use -u option. The following will show only the top processes that belongs to oracle user.
$ top -u oracle

25. df command examples

Displays the file system disk space usage. By default df -k displays output in bytes.
$ df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             29530400   3233104  24797232  12% /
/dev/sda2            120367992  50171596  64082060  44% /home
df -h displays output in human readable form. i.e size will be displayed in GB’s.
ramesh@ramesh-laptop:~$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1              29G  3.1G   24G  12% /
/dev/sda2             115G   48G   62G  44% /home
Use -T option to display what type of file system.
ramesh@ramesh-laptop:~$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext4    29530400   3233120  24797216  12% /
/dev/sda2     ext4   120367992  50171596  64082060  44% /home

26. kill command examples

Use kill command to terminate a process. First get the process id using ps -ef command, then use kill -9 to kill the running Linux process as shown below. You can also use killall, pkill, xkill to terminate a unix process.
$ ps -ef | grep vim
ramesh    7243  7222  9 22:43 pts/2    00:00:00 vim

$ kill -9 7243

27. rm command examples

Get confirmation before removing the file.
$ rm -i filename.txt
It is very useful while giving shell metacharacters in the file name argument.
Print the filename and get confirmation before removing the file.
$ rm -i file*
Following example recursively removes all files and directories under the example directory. This also removes the example directory itself.
$ rm -r example

28. cp command examples

Copy file1 to file2 preserving the mode, ownership and timestamp.
$ cp -p file1 file2
Copy file1 to file2. if file2 exists prompt for confirmation before overwritting it.
$ cp -i file1 file2

29. mv command examples

Rename file1 to file2. if file2 exists prompt for confirmation before overwritting it.
$ mv -i file1 file2
Note: mv -f is just the opposite, which will overwrite file2 without prompting.
mv -v will print what is happening during file rename, which is useful while specifying shell metacharacters in the file name argument.
$ mv -v file1 file2

30. cat command examples

You can view multiple files at the same time. Following example prints the content of file1 followed by file2 to stdout.
$ cat file1 file2
While displaying the file, following cat -n command will prepend the line number to each line of the output.
$ cat -n /etc/logrotate.conf
    1 /var/log/btmp {
    2     missingok
    3     monthly
    4     create 0660 root utmp
    5     rotate 1
    6 }

31. mount command examples

To mount a file system, you should first create a directory and mount it as shown below.
# mkdir /u01

# mount /dev/sdb1 /u01
You can also add this to the fstab for automatic mounting. i.e Anytime system is restarted, the filesystem will be mounted.
/dev/sdb1 /u01 ext2 defaults 0 2

32. chmod command examples

chmod command is used to change the permissions for a file or directory.
Give full access to user and group (i.e read, write and execute ) on a specific file.
$ chmod ug+rwx file.txt
Revoke all access for the group (i.e read, write and execute ) on a specific file.
$ chmod g-rwx file.txt
Apply the file permissions recursively to all the files in the sub-directories.
$ chmod -R ug+rwx file.txt

33. chown command examples

chown command is used to change the owner and group of a file. \
To change owner to oracle and group to db on a file. i.e Change both owner and group at the same time.
$ chown oracle:dba dbora.sh
Use -R to change the ownership recursively.
$ chown -R oracle:dba /home/oracle

34. passwd command examples

Change your password from command line using passwd. This will prompt for the old password followed by the new password.
$ passwd
Super user can use passwd command to reset others password. This will not prompt for current password of the user.
# passwd USERNAME
Remove password for a specific user. Root user can disable password for a specific user. Once the password is disabled, the user can login without entering the password.
# passwd -d USERNAME

35. mkdir command examples

Following example creates a directory called temp under your home directory.
$ mkdir ~/temp
Create nested directories using one mkdir command. If any of these directories exist already, it will not display any error. If any of these directories doesn’t exist, it will create them.
$ mkdir -p dir1/dir2/dir3/dir4/

36. ifconfig command examples

Use ifconfig command to view or configure a network interface on the Linux system.
View all the interfaces along with status.
$ ifconfig -a
Start or stop a specific interface using up and down command as shown below.
$ ifconfig eth0 up

$ ifconfig eth0 down

37. uname command examples

Uname command displays important information about the system such as — Kernel name, Host name, Kernel release number,
Processor type, etc.,
Sample uname output from a Ubuntu laptop is shown below.
$ uname -a
Linux john-laptop 2.6.32-24-generic #41-Ubuntu SMP Thu Aug 19 01:12:52 UTC 2010 i686 GNU/Linux

38. whereis command examples

When you want to find out where a specific Unix command exists (for example, where does ls command exists?), you can execute the following command.
$ whereis ls
ls: /bin/ls /usr/share/man/man1/ls.1.gz /usr/share/man/man1p/ls.1p.gz
When you want to search an executable from a path other than the whereis default path, you can use -B option and give path as argument to it. This searches for the executable lsmk in the /tmp directory, and displays it, if it is available.
$ whereis -u -B /tmp -f lsmk
lsmk: /tmp/lsmk

39. whatis command examples

Whatis command displays a single line description about a command.
$ whatis ls
ls  (1)  - list directory contents

$ whatis ifconfig
ifconfig (8)         - configure a network interface

40. locate command examples

Using locate command you can quickly search for the location of a specific file (or group of files). Locate command uses the database created by updatedb.
The example below shows all files in the system that contains the word crontab in it.
$ locate crontab
/etc/anacrontab
/etc/crontab
/usr/bin/crontab
/usr/share/doc/cron/examples/crontab2english.pl.gz
/usr/share/man/man1/crontab.1.gz
/usr/share/man/man5/anacrontab.5.gz
/usr/share/man/man5/crontab.5.gz
/usr/share/vim/vim72/syntax/crontab.vim

41. man command examples

Display the man page of a specific command.
$ man crontab
When a man page for a command is located under more than one section, you can view the man page for that command from a specific section as shown below.
$ man SECTION-NUMBER commandname
Following 8 sections are available in the man page.
  1. General commands
  2. System calls
  3. C library functions
  4. Special files (usually devices, those found in /dev) and drivers
  5. File formats and conventions
  6. Games and screensavers
  7. Miscellaneous
  8. System administration commands and daemons
For example, when you do whatis crontab, you’ll notice that crontab has two man pages (section 1 and section 5). To view section 5 of crontab man page, do the following.
$ whatis crontab
crontab (1)          - maintain crontab files for individual users (V3)
crontab (5)          - tables for driving cron

$ man 5 crontab

42. tail command examples

Print the last 10 lines of a file by default.
$ tail filename.txt
Print N number of lines from the file named filename.txt
$ tail -n N filename.txt
View the content of the file in real time using tail -f. This is useful to view the log files, that keeps growing. The command can be terminated using CTRL-C.
$ tail -f log-file

43. less command examples

less is very efficient while viewing huge log files, as it doesn’t need to load the full file while opening.
$ less huge-log-file.log
One you open a file using less command, following two keys are very helpful.
CTRL+F – forward one window
CTRL+B – backward one window

44. su command examples

Switch to a different user account using su command. Super user can switch to any other user without entering their password.
$ su - USERNAME
Execute a single command from a different account name. In the following example, john can execute the ls command as raj username. Once the command is executed, it will come back to john’s account.
[john@dev-server]$ su - raj -c 'ls'

[john@dev-server]$
Login to a specified user account, and execute the specified shell instead of the default shell.
$ su -s 'SHELLNAME' USERNAME

45. mysql command examples

mysql is probably the most widely used open source database on Linux. Even if you don’t run a mysql database on your server, you might end-up using the mysql command ( client ) to connect to a mysql database running on the remote server.
To connect to a remote mysql database. This will prompt for a password.
$ mysql -u root -p -h 192.168.1.2
To connect to a local mysql database.
$ mysql -u root -p
If you want to specify the mysql root password in the command line itself, enter it immediately after -p (without any space).

46. yum command examples

To install apache using yum.
$ yum install httpd
To upgrade apache using yum.
$ yum update httpd
To uninstall/remove apache using yum.
$ yum remove httpd

47. rpm command examples

To install apache using rpm.
# rpm -ivh httpd-2.2.3-22.0.1.el5.i386.rpm
To upgrade apache using rpm.
# rpm -uvh httpd-2.2.3-22.0.1.el5.i386.rpm
To uninstall/remove apache using rpm.
# rpm -ev httpd

48. ping command examples

Ping a remote host by sending only 5 packets.
$ ping -c 5 gmail.com

49. date command examples

Set the system date:
# date -s "01/31/2010 23:59:53"
Once you’ve changed the system date, you should syncronize the hardware clock with the system date as shown below.
# hwclock –systohc

# hwclock --systohc –utc

50. wget command examples

The quick and effective method to download software, music, video from internet is using wget command.
$ wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.1.tar.gz
Download and store it with a different name.
$ wget -O taglist.zip http://www.vim.org/scripts/download_script.php?src_id=7701

Ansible: Roles

Use Ansible roles to orchestrate more complex configurations.Let's create a new directory named  nginx , which will be a Role. Then we&...