Tutorial on TRAP in Unix

Rahul / Linux / 0 Comments and 0 Reactions

Some times in Unix we want to prevent our script from abnormal conditions and interrupts, this can be handled using the trap command in Unix.When you press the Ctrl+C or Break key at your terminal during execution of a shell program, normally that program is immediately terminated, and your command prompt returned. This may not always be desirable. For instance, you may end up leaving a bunch of temporary files that won’t get cleaned up.

Trapping these signals is quite easy, and the trap command has the following syntax:

Format: trap command signal

Trap enables signals handling ,signal are sent to running process to indicate that an event (external to the process) has occurred that the process must respond,Now, trap catches a signal sent to the process, and take action on the signal using the action defined in the trap command, Instead of letting the signal take it’s default effect on the process.Command would be one or more command that will be executed (separated by colon) when any of specified signal received.

Signal: The signal we want to trap , we can specify more than one signal here, it can be both numeric or string (signal name) format as described below.

0 EXIT Exit
1 HUP Hang-up
2 INT Interrupt (like ctrl +c)
15 TERM Software termination signal

While scripting we might get confuse where we put our trap command, trap command executed whenever one of the traps on its list occurs,This means that our shell needs to know in advance what to do about a signal before a signal occurs.This advice that we have our trap command before any otherexecutable command, In this case the shell will be able to know which signal to send our trap command.

Like : trap 1………
Command 1
Command 2
Command 3
trap 2 ………………

Command 4
Command 5
Command 6

Here, trap 1 will be executed for command 1,2,3, and trap 2 will be executed for command 4,5,6.
Now we can define multiple traps in our program 

Usage of trap : Suppose we want to prevent our program from hang-up and interrupt signals.trap ‘echo “you program interrupted”;rm /tmp/daemons1.flg;exit 1’ 1 2 .Here it will echo the msg, remove some log or flag file according to our requirement and then exit, once it receive 1 or 2 signal,Sometimes we might want to ignore certain signals while performing some operation that we don’t want interrupted,

trap ‘ ‘ 1 2

Here we are ignoring 1 and 2 signal by providing empty command in trap.We can also defined function in trap, if we have much command to execute,trap ‘function_name;exit1’ INT HUP

Change the trap result back for signals We can change the default action of a signal, like trap 1 2

NOTE : If you ignore a signal, all subshells also ignore that signal. However, if you specify an action to be taken on receipt of a signal, all subshells will still take the default action on receipt of that signal.The commands specified to trap must be enclosed in quotes if they contain more than one command. Also note that the shell scans the command line at the time that the trap command gets executed and also again when one of the listed signals is received.

Tutorial on SSH keys and Their Permissions in Unix

Rahul / Linux / 0 Comments and 1 Reaction

Tutorial on SSH keys and Their Permissions in Unix.The conventional protocols (telnet, ftp, rlogin, X) used to connection computers on the Internet are insecure.For example, if you use telnet, your password may be “sniffed” by an intruder on any network between the client and server.A new version of sshwas created (ssh2). This has a more restrictive license, so most places only use the original version (ssh1), and today we will talk about ssh1 keys.

1) Run ssh-keygen as follows  $ ssh-keygen ,   Enter file in which to save the key (~/.ssh/identity): [RETURN] ,   Enter passphrase: [ENTER A GOOD PASS PHRASE TO PROTECT THE PRIVATE KEY]  ,   Enter the same passphrase again: [DITTO] , This creates a private and public key pair in ~/.ssh/identity and ,~/.ssh/identity.pub. The first file is protected by (a) Unix permissions,and (b) your pass phrase. The second file only needs to be protected.against writing by anyone except you.

NAME :ssh-keygen - authentication key generation, management and conversion

SYNOPSIS

ssh-keygen [-q ] [-b bits ] -t type [-N new_passphrase ] [-C comment ] [-f output_keyfile ]

ssh-keygen -p [-P old_passphrase ] [-N new_passphrase ] [-f keyfile ] ssh-keygen -i [-f input_keyfile ] ssh-keygen -e [-f input_keyfile ] ssh-keygen -y [-f input_keyfile ] ssh-keygen -c [-P passphrase ] [-C comment ] [-f keyfile ] ssh-keygen -l [-f input_keyfile ] ssh-keygen -B [-f input_keyfile ] ssh-keygen -D reader
ssh-keygen -U reader [-f input_keyfile ]

Normally this program generates the key and asks for a file in which to store the private key. The public key is stored in a file with the same name but “.pub” appended. The program also asks for a passphrase. The passphrase may be empty to indicate no passphrase (host keys must have an empty passphrase), or it may be a string of arbitrary length. A passphrase is similar to a password, except it can be a phrase with a series of words, punctuation, numbers, whitespace, or any string of characters you want. Good passphrases are 10-30 characters long, are not simple sentences or otherwise easily guessable (English prose has only 1-2 bits of entropy per character, and provides very bad passphrases), and contain a mix of upper and lowercase letters, numbers, and non-alphanumeric characters. The passphrase can be changed later by using the -p option.

mkdir ~/.ssh chmod 700 ~/.ssh ssh-keygen -t rsa

You will be prompted for a location to save the keys, and a passphrase for the keys. This passphrase will protect your private key while it’s stored on the hard drive and be required to use the keys every time you need to login to a key-based system:

Generating public/private rsa key pair. Enter file in which to save the key (/home/b/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/b/.ssh/id_rsa. Your public key has been saved in /home/b/.ssh/id_rsa.pub.

Your public key is now available as .ssh/id_rsa.pub in your home folder.Permission problems with SSH

ssh is very picky about permissions on the ~/.ssh directory and files. Sometimes you may do something to mess up these permissions. Run the following to fix most permissions problems. You may have to do this on both the remote host and local host.

chmod 700 ~/.ssh

chmod 600 ~/.ssh/id_rsa

chmod 644 ~/.ssh/id_rsa.pub

chmod 644 ~/.ssh/authorized_keys

chmod 644 ~/.ssh/known_hosts

Also no directory above ~/.ssh can have ‘group’ or ‘other’ write permissions.Note : If  you give permissions greater than these you will get errors like “too much permissions”.

Tutorial on Deleting Metacharatered Files

Rahul / Linux / 0 Comments and 1 Reaction

Tutorial on Deleting Metacharatered Files

If you’ve transferred files to your Unix account from a PC or Macintosh with filenames containing what Unix considers to be meta-characters, they may cause problems. Meta-characters (including semicolons, spaces, backslashes, dollar signs, question marks, and asterisks) are characters that are interpreted under Unix as commands or instructions. Although these characters may not cause any trouble in other operating systems, their special Unix interpretations may cause problems when you try to delete them. Lets start with how to delete metacharactered files in Unix Environment , Sometimes accidentally, we create a strange file that can’t be deleted because of it being created with space or some control characters in its name,

1: One of the best way to remove the file using the inode number  , Here list is a metacharactered file, we can get the inode number of file by running ls –lirt (where i stand for inode no.)

546688 -rw-rw-r– 1 username groupname 39 Oct 9 19:53 sftdvis
546869 -rw-rw-r– 1 username groupname 55 Oct 9 21:23 dvis
96015 -rwxrwxr-x 1 username groupname 1637 Dec 16 14:18 TestScript.sh
114500 -rw-r–r– 1 username groupname 8512 Dec 27 12:48 PaymentDD.class
115912 -rw-r–r– 1 username groupname 22355 Dec 29 13:06 csAPIInBan_Dl.class
546533 drwxrwxr-x 2 username groupname 96 Jan 10 06:06 test
546557 -rw-rw-r– 1 username groupname 0 Jan 10 06:10 list

Then running the find command to delete that file,
find . –inum 546557 –exec rm {} \; ( inum represent the inode number of that file )

2 :There is also another quick way,might work most of the time,Just list the metacharactered files using ls –lrqt (where q list nonprinting character in the file name),

lrwxrwxrwx 1 username groupname 27 Mar 21 2011 abp_home -> /ENBSTSTR/vdc/ST_ST_SP6_P26
drwxr-xr-x 8 username groupname 8192 May 11 2011 pbin_V64
drwxrwxr-x 3 username groupname 8192 May 11 2011 patch
-rw-r–r– 1 username groupname 5933 Jul 29 12:32 VDC9I
-rwxr-xr-x 1 username groupname 367 Aug 1 14:23 a.sh
drwxrwxr-x 4 username groupname 96 Aug 23 09:11 Patches
-rw-rw-r– 1 username groupname 39 Oct 9 19:53 sftdvis
-rw-rw-r– 1 username groupname 55 Oct 9 21:23 dvis
-rwxrwxr-x 1 username groupname 1637 Dec 16 14:18 TestScript.sh
-rw-r–r– 1 username groupname 8512 Dec 27 12:48 PaymentDD.class
-rw-r–r– 1 username groupname 22355 Dec 29 13:06 csAPIInBan_Dl.class
drwxrwxr-x 2 username groupname 96 Jan 10 06:06 test
-rw-rw-r– 1 username groupname 0 Jan 10 06:20 listst??

Then run rm listst?? , it will remove that file.

3.There is another way if there is spaces in the file name , Rm –rf “file name with spaces”.To remove a file which have spaces in between its name.Another format:rm file\ with\ spaces\ in\ the\ name .Use quotes to surround the file name:rm “file with a space.txt”.

4.To remove a file whose name begins with a dash ( - ) character, refer to the file with the following syntax: rm ./-filename.Some Emacs editors allow you to directly edit a directory, and this provides yet another way to remove a file with a troublesome name.

5.There are some characters that you cannot remove using any of the above methods, such as forward slashes, interpreted by Unix as directory separators. To remove a file with such meta-characters, you may have to FTP into the account containing the file from a separate account and enter the command: mdel

You will be asked if you really wish to delete each file in the directory. Be sure to answer  n  (for no) for each file except the file containing the difficult character that you wish to delete. Delete that file by typing  y  (for yes) when prompted. Once you’ve deleted the intended file, you may press Ctrl-c to discontinue the mdel process. Use this approach cautiously to avoid deleting other files by mistake.

Six Simple And Awesome Awk One Liners for Unix Interviews

Rahul / Linux, Programming / Answers, Awk, Interviews, Questions, unix / 0 Comments and 0 Reactions

Awk is a little programming language, with a syntax close to C in many aspects. It is an interpreted language and the awk interpreter processes the instructions. Today I present to you some tricks which can Make our lives really easy in Unix.

1.To Negate some lines with AWK:

 These are called awk one liners. The First trick for today is how to negate some rows while using awk.

The Trick is : awk -F ” ” ‘NR>5 && NR<7{print $1,$2,$3}’ 1.txt

Note : This question will mostly asked in interviews where you will be asked to print some parameters from a file mostly a log file where in you will be asked to negate some rows herein refered to as records in awk. The above command here performs the following action. The field separator mostly called as delimiter here will be a space and then it will negate all the records and will only iterate through line numbers five through seven and then print the first second and third parameter.

2. To Take only a few Lines:

How would you print only lines match between Rahul and Abhishek using awk? Considering your file names.txt contains :
Rahul
Sahil
Geetika
Mamta
Abhishek
and the expected output should be like :
Sahil
Geetika
Mamta

The Trick is : $ awk ‘/Rahul/{ P=1; next } /Abhishek/ {exit} P’ names.txt

3. Context Addressing in AWK:

The above command will give you the desired output. Just try. If the Interviewer asks you what is this called, Just say this is called the CONTEXT addressing in awk. The interviewer might also be interested in asking you to print a certain range of lines from a file where in you do not have a pattern defined. We all know awk works on the principle of condition { actions }. Where condition is typically an expression and action is a series of commands. The input is split into records, where by default records are separated by newline characters so that the input is split into lines. So we know the context addressing can be used just like grep, so here you go.

The Trick is : awk ‘$0 ~ /pattern/ {print $0}’

4. Taking a range of records:

Suppose one wants to print all the lines in a file that match some pattern, the above code should be used.Lets suppose he wants you to print logs from line number 15000 to 16000.Just give him a simple blunt reply.

The Trick is  : head -16000 filename | tail -1000.

5. Deleting only files :

Now the interviewer might be tempted to ask you to delete all the files in the current directory with awk.

The Trick is  : ls -l|awk ‘$1!~/^drwx/{print $9}’|xargs rm

6. Removing Duplicates without sorting :

Be careful when trying this out in your home directory. This will remove all the files. awk syntax is not the same in every Unix system, but there is a way to learning how is it in our particular system: man awk. Now lets see our last trick, this talks about deleting the duplicate entries in file without sorting. There are thousand ways of doing one thing in unix lets see a couple of them. Suppose your file text.txt has contents the below contents.

/users/env> cat test.txt
abcd
efgh
ijkl
mnop
abcd
/users/env > cat test.txt |awk !’x[$0]++’
abcd
efgh
ijkl
mnop

Now this one lines removes the duplicate without sorting them. Enjoy Playing with awk.

10 must know find and replace command in UNIX

Rahul / Linux, Programming / commands, find, must know, replace, unix / 0 Comments and 2 Reactions

In Unix often we are asked to perform certain things which requires us to find a particular pattern and then to play or manipulate that particular pattern, so now today let us learn how to be a master in that. The list here present 10 must know find and replace command in Unix with explanation.

1. Finding any extension files:

Our first example is how to find a set of files and then play with the files, to play we mean to manipulate the files. The first example deals with finding files which are older than certain time or heavier than certain, we do this most often with log files. Now lets find files which are 15 days old.

We can find files with any extension with *.* feature and then we can find with respect to what option we use with it, for example we will use the mtime option here, mtime is for modification time.

find *.* -mtime -15

find does the recursive search as well via find / -name file name we know that / searches from the root.

2. Finding particular set of files:

find . -name “*.c” -print.

. (dot) searches in current directory and will give relative path.*.c will give only the c extension files. Finding files which are one year old will be found by the below command, so the files which are not accessed from a year will be as below.

3. Finding old files with access time:

Find $HOME will search in the home directory

find /home -atime +365 -print

4. Finding and  negating:

Now the below command will select everything but the c files, so this option negate what we specify and find everything other than that.

find .! -name “*.c” -print

5. Other options in Find:

Now we can also use options with the find command. For example the size option will find the files in order of its size, the inum with the inode number details and type f with finding only the files. Example :

-size +2048 ;    -inum 12345 ;    -type f.

6. Deleting old files:

Now as we have already found out the files we will play with them, lets delete them first. This will be majorly helpful when we want to delete the old logs which are a month old. In this example we are deleting files older than 15 days.

find *.* -mtime -15 -exec rm -rf {}\;

7. Finding a pattern:

Now if we want to find a particular pattern in all files using the find command we can certainly use it to our ease. In the below example we will find the pattern rahul.

find . -type f -exec grep “rahul” {} \; -print

8. Finding and replacing:

Now we can replace the pattern we have found with multiple options, one of them is using perl. In the below example we will replace the pattern rahul with pattern miglani using perl.

find /path/to/start/from/ -type f | xargs perl -pi -e ‘s/rahul/miglani/g’

9. Finding and replacing in multiple files:

The below example finds the pattern rahul in all text files and then in all those files replaces the patternrahul with the pattern miglani with the use of strean editor. The grep -il statement finds all instances of rahul in files ending in txt and i = ignore case, l = only list the filename. The file names are passed to sed, which runs a regular expression to change all instances of rahul to miglani. Since sed doesn’t overwrite a file, I redirected the output to a temp file and then renamed it back to the original file name.

#!/bin/sh
for file in $(grep -il “Rahul” *.txt)
do
sed -e “s/Rahul/Miglani/ig” $file > /tmp/tempfile.tmp
mv /tmp/tempfile.tmp $file
done.

10. Deleting the carrot M character:

Now our very own find can also be used to t replace ^M character which is introduced in our files when we export files from windows to Unix. Now the below example uses a simple loop to do the same.

for i in `find . -type f` ; do dos2unix $i $i; done

Enjoy playing with files and patterns  .

AWK

AWK- an advanced fileter

aho,weinberger and kernighan

gawk in linux

awk options 'selection criteria {action}' files

printing is the default action of awk

awk '/rahul/' 1.txt
awk '/rahul/{print}' 1.txt

both are same and will print the lines containing rahul

awk -F" " '/rahul/{print $2,$3}' 1.txt
it will print 2nd and 3rd parameter in all lines containing rahul delimited by a space

 awk -F " " 'NR==1,NR==10{print $1,$2,$3}' 1.txt
it will print first 10 lines ;its 1st , 2nd and third parameters delimited by space.

awk 'BEGIN{FS=" "}
=> NF!=6{
=> print "record number",NR,"has",NF,"fields"}' 1.txt

using built in variables

NR no of records
FS field separator
OFS output fiels separator
NF Number of fields in current line
FILENAME current input file
ARGC no of arguments in current line
ARGV List of arguments


using functions

awk -F" " 'length($2) < 1' 1.txt

those which have short second parameter