Monday, December 27, 2010

Using sed to delete line with text containing file path

Hi

It took me a day to figure out, how can i delete a line in file using sed. This was pretty simple.

#sed 's/your_string//g'

Now if your string is in a variable than you are going to use is.


#sed 's/$var//g'

**Now problem was your variable contains a path of a file 'say /etc/passwd '
It really doesn't works simply. because of '/' constraints. 

I figured it out to be worked as....

sed -i "\#$i#d" $infile

This way value in variable $i will be deleted from file "$infile" 
This is not very easily available on net.




Some more sed important examples...
-> To delete blank line from a text file.
sed '/^$/d'

This is to remove all spaces from the file.

sed 's/ //g'

Sunday, December 5, 2010

Nayak Movie Anil Kapoor

Today i was watching Nayak Movie on one of the TV Channel one of my favorite movie. This is one movie shown again and again on TV, I am sure that people like to watch this movie.

People like to watch movie which show their dreams on screen. We badly want to see same thing happening in real life. We are waiting for a hero like Anil Kapoor to stand up and lead us to a bright future including me.

How do you feel when Anil Kapoor suspends all corrupt officers, how you feel when you see him talking to delegations for implementing new projects. :)

will continue .........busy watching Peepli Live right now .. 

Monday, November 15, 2010

What is Load Average

Understanding what is load average can be very confusing when you read man pages and other documents. So here is my understanding about load average.

Think about a bridge from where 100 cars can pass through at a time. So in load average context 100 processes can process at a time.

Now if there are only 50 cars going from that bridge that means 100:50 Now load average is .50 if there are 150 cars than 50 cars will be waiting to pass through that bridge in that context load average will 100:150 ie. load average of 1.50.

Hope it is clear in a single processor context. Now what if we have dual core processor or quad processor. In our example lets talk about quad processor.

Now there are 4 bridges which a capacity of 100 cars each bridge. In this context if have 150 cars coming than load average will be 1. This might be confusing as i think there are static calculation methods . Average will still come as 1.5 but now  we still have spare capacity of 250 cars. So it would be fine to have load average of 1.5 which was not good in case of a single core processor.

In a quad core processor it is fine to have a load average anything below 4 , but can be a problem if it goes above 4 as that will means more than 400 cars for a bridge with capacity of 400 cars.

so why don't we have some standard for load average without looking at number of CPU's my machine, I don;t know about this as of now, but will post the same here.

Thursday, November 11, 2010

SVN UN-delete

If you want to recover any folder or file which was deleted accidentally  you can recover the same. First you need to figure out the revision number when it was deleted. Now use this command on server to get back file/folder .

Remember even if you perform delete operation in SVN nothing gets deleted it is just omitted from future checkout's and updates.


 svn cp -r 47 https://amit.example.com/svn/repos/test_repo/data/somefolder/Debug  https://amit.example.com/svn/repos/test_repo/data/somefPublish Postolder/

Sunday, November 7, 2010

Code Freeze SVN

To implement Code freeze in SVN you can write pre-commit hook script. Here is a sample script which does this job on specified paths. You please remove all echo in case implementing on production.


REPOS="$1"
TXN="$2"

echo "value of REPOS is :$REPOS"  >> /svnroot/test_repo_precommit/hooks/precommit.log
echo "value of TXN is :$TXN"  >> /svnroot/test_repo_precommit/hooks/precommit.log

REPO_URL_TO_MATCH=(
                '/svnroot/test_repo_precommit/somepath1'
'/svnroot/test_repo_precommit/somepath2'
'/svnroot/test_repo_precommit/somepath2'
);

echo "printing REPO_URL_TO_MATCH array values : ${REPO_URL_TO_MATCH[@]}"  >> /svnroot/test_repo_precommit/hooks/precommit.log

for REPO_URL_TO_MATCH in ${REPO_URL_TO_MATCH[@]}
do

echo  "in for loop print REPO_URL_TO_MATCH :$REPO_URL_TO_MATCH" >> /svnroot/test_repo_precommit/hooks/precommit.log;

RESULT1=`svnlook dirs-changed "$REPOS" -t $TXN | grep -e "$REPO_URL_TO_MATCH"`;

echo " printing value of RESULT1 : $RESULT1"  >> /svnroot/test_repo_precommit/hooks/precommit.log;

for pattern in $RESULT1
do
 if [ ! -z "$pattern" ];
echo " Value of pattern is :$pattern " >> /svnroot/test_repo_precommit/hooks/precommit.log

then
echo "*** Your commit has been blocked because code is in freezed mode, Please contact repo administrator to remove code freeze from this path."
exit 1
fi
done
echo "Exit from Loop one " >> /svnroot/test_repo_precommit/hooks/precommit.log

done
echo `date` >> /svnroot/test_repo_precommit/hooks/precommit.log
echo "*********************This is end of script************************************" >> /svnroot/test_repo_precommit/hooks/precommit.log

Tuesday, October 12, 2010

What a job

As a System Administration you will work with world class teams and lead system engineering and administration efforts of our globally distributed, Web 2.0 products used by millions worldwide. These products involve myriad technologies, diverse platforms, Server load balancing, High Availability (HA) clusters, IP/FC SANs, Distributed File Systems etc.
In any of these roles, you will lead a team of highly technical System Administrators and will be responsible for the architecture, planning, implementation, configuration, automation, deployment, administration, monitoring and management of our large-scale, globally distributed products. We are sticklers for automation, and as such you will be expected to automate all processes using myriad scripting languages, automation tools, open source software etc. You will be involved in macro decisions as well as actual implementations in the trenches along with your troops. You will be expected to keep abreast of the latest and greatest in system administration. Your role will also involve recruitment, training, mentoring and knowledge-sharing.
Some of your responsibilities will include -
Auto-Installation, Configuration Management, High Availability, Clustering, Load Balancing, Planning Storage Systems, Monitoring, Reporting, Data migration, Replication, Backups, Networking, Equipment Selection, Security Assessments, Troubleshooting, Managing DNS, Delivering 5 nines Uptime, Scalability, Managing logging and troubleshooting, Scripting, Research, Software Selection, Platform selection, Architecture, Performance tuning, Automated package deployment, Auto-updates, Server hardening etc
FOSS experts, Unix/Linux gurus, Uber geeks/nerds. You love phrack, ;login & miss Sysad magazine. You use mutt for email, run bleeding edge distros/softwares, use NNTP/IRC/mailing lists, enjoy & contribute to postfix vs qmail vs send mail vs Exim vs Exchange OR vi/vim vs emacs debates, are an avid blogger, and believe that a shell is the only application one needs for performing any task.
You have 8+ years experience in System Architecture and Administration of which 2+ years have been spent in managing a reasonably sized team of highly technical System Administrators managing 500+ nodes using automated processes and systems.
You have spent a considerable amount of time bootstrapping large distributed infrastructure deployments from scratch and have gained significant exposure to high availability, scalability, architecture, problem solving and troubleshooting skills. You are extremely tech focused - a configuration management and automation guru. You NEVER do repeatable stuff twice and have created a repo of timeless automation scripts/configuration management code over the years. You possess sharp reasoning and logical skills, are a master of regular expressions, have a good grasp on programming traditions/tools/languages and can whip a script/program as required. You use gdb/SystemTap/Frysk to debug system problems/coredumps/backtraces.
You are a DNS & networking guru with the ability to troubleshoot networking issues with tcpdump/wireshark/etc. You are not rigid with respect to choice of technology, are relatively platform agnostic, and open to adopting the right technology for the task at hand. You have exceptional oral/written communication skills and a good sense of business.
You are innately curious, and take time out to keep up with system administration trends and practices. You can deliver reliable systems, monitoring tools, scripts and application configurations at a rapid pace.
Skills:
You are acquainted with / have mastered a few of the below skills/tools -
Linux/FreeBSD/OpenBSD/Solaris, Any Shell/Bash, PERL/C/Ruby/Python/PHP, Basic DNS & Networking, TCP/UDP, IP Routing, tcpdump, netcat, netstat, lsof, ethereal/wireshark, IPTables, Nagios, Cactii, RRDTool, MRTG, SNMP, Postfix/qmail/Exim/Sendmail, RDBMS, MySQL/MySQL Cluster, Postgres, MSSQL, BDB, IPC, Regular expressions, Caching, nginx/perlbal/varnish/squid, Perdition, Apache/lighttpd/Axes/IIS, Tomcat/Resin, Rewrite, SSH/SSL, LDAP/Active Directory, RADIUS, DHCP, Kickstart/Cobbler, Yum/Up2date, Stateless Linux, NIS, NFS/CIFS, ext3/ext4/GFS/XFS/ZFS/btrfs, DRBD/NBD, RAID, IP/FC SAN, NAS, Heartbeat/Wackamole, LVS/ldirectord, Load Balancing, Multi-threading, Concurrency, Async IO, Scalability, Emacs, VI/VIM, KVMs, gdb, frysk, SystemTap, oprofile, GIT, CVS, SVN... ad infinitum

Monday, September 27, 2010

Institute of political Leadership

Hi

This post is just to remember that there is institute for political leadership. :)

cool isn't it.

http://www.iplindia.in/

Sunday, September 12, 2010

Lives we are making Simple / Complicated

I have to go to office in morning and i am writing this post as i am not able to sleep :(


Have you ever thought that why we work.......other than Salary .. What value we are adding to this world by working. May be are trying to make things better and simpler, and in order to do that in most efficient (in term of cost\time ) we are making things more complex.

I am a System Admin, right now handling a Application support.  Application which supports developers to manage there code and help project manager to plan for projects.

These developers are writing code for their clients may be banking , telecom , those companies are again working to provide me better services and see what, indirectly am doing all for my comfort.

This circle made me understand a lot of things that if i will not give good service i am not going to get good service. 


But am i really in comfort am i actually enjoying life, am i really earn to live, seems like i am living to earn.
Life seems to be like an race, we humans are again living like animals prisoners to our schedule.

Some time i really think about life in past, we have been trying to make out lives better, are we successful enough to do this, are we living a better life, writing this blog sitting on my laptop gives me feel definitely things have changed and are much better from past, than why it happens that feeling of fulfillment is not there, there are something missing, something incomplete.

If only i was again a 6 Year old child and make excuse of stomach ace for not going to school. :(

Wednesday, September 8, 2010

Start a Political party

Here are the things which can be implemented to start a political party. I am writing this because of my low memory. Idea's flash in my mind and than they disappear as they were never there.  I as such don't have intention of starting a political party but i think things mentioned over here are surly going to work to win you at least one election later on all depends upon the work you do.

=> People are fed of system and 50% of India doesn't VOTE. NEVER ASK THEM TO VOTE FOR YOU rather ask them to just VOTE. Start a campaign in which ask people to VOTE, ask them to join you in movement to choose a right candidate and there should be a option of choosing NO candidate. Change and reforms in Elections is 1st thing, bring this in front of people. In your campaign publish 49-O as much as possible.

try to convince people not to give vote in influence of liquor or money offered.
=> Never made any promises, people don't have faith in promises.
=>Stand against reservation this will bring a big vote bank. tell them you will not give them bread but skills by which they can earn bread.
=>Publish bio-data of your candidates not the pictures, tell them how successful they already are in there lives, you need to convey that they are not in politics for money and power. Publish there wealth assets on your website.

=>ask for donation and funds to make them believe that you don't have corporate support with you.
=> Pitch for change in there social life conditions.
=> Bring issue that how politicians make rules and laws for common men which do not implement on them. Bring example of BRT corridor. Tell them your party workers will move in same lanes in which they are moving.
=> Justice delayed is Justice denied - tell them you are going to bring reforms in our courts. (THIS IS THE BIGGEST POINT TO CASH ON)
=>RTI will be available over phone.
=> your local constituency expenses made available to all in a form of news paper to all once in 6 months.
=>MCD complaints will handled over phone, all pit holes to filled in 48 hrs of reporting.
=> bring as much as local issues to pictures which effect the common men directly.
=>Common men (Middle class) people are the key to WIN elections.
=> Just don't tell what are you going to do tell people how you are going to do that. (People don;t believe un-realistic promises)

* You need to find people who can fund you for TV advertisement.
* You need to find people you will stand with you fight elections with you.

Thursday, September 2, 2010

Monday, August 23, 2010

blocking telnet on active port

 Telnet can connect to any server that utilizes TCP on any port by simply specifying the desired port after the address.  This can be quite useful for troubleshooting, as you can see the raw reply (or lack of connection) without the interpretation of the typical client.

The short answer is no, it is not possible to DIRECTLY block this type of connection.  It is possible to block some types of traffic based upon the IP protocol being used.  A common example is to block ping requests by refusing ICMP traffic.  In this case, ICMP is the protocol used over IP.  The IP header has a flag that indicates the traffic is ICMP and the type of ICMP request.  Either or both of these can be examined by a
firewall to determine what action should be taken, so it is a very simply matter to drop or refuse all ICMP traffic or just ping requests.

Telnet and HTTP both use TCP protocol over IP (as well as many other services).  There is nothing in a telnet header that distinguishes it from a HTTP header or any other TCP traffic.  You must either allow TCP traffic to a port or deny it.  With this in mind, if you want to allow HTTP traffic on port 80, you must accept all TCP connections on that port.

Once the connection is made, there are some things in the actual data that would indicate that the connection is coming from a telnet client instead of a http client.  Most notably, the initial data from a telnet client will be sent one character per packet, maybe two if you type really fast.  In contrast, a traditional http client will sent complete http request in one or two packets starting with the third packet of the connection.  

It would be fairly simple to detect the small packets with some firewalls and kill the connection.  This is not really blocking the connection, as you would have to allow the initial connection to get at the subsequent data, but it would have the effect of disallowing connections from a telnet client.  Another method would be to filter the first few packets and kill connections that do not have some of the key elements of a proper http request.

A better method would be to configure the http server to refuse connections from clients that do not provide a proper http header.  It certainly is possible to send a reasonable header from telnet, but it would involve a bit of very accurate typing.  Cumbersome at the very least.  In this case, as with the other method the TCP connection is technically allowed, but nothing in the way of meaningful data is passed.

I don't really see much point in blocking this type of traffic.  Telnet to a non-telnet server can be a useful diagnostic tool, not much use for anything else as the connection is closed after each exchange.





_______________________________________


"Hi, is it possible to block telnet access to a port that is eg listening on port 80."

No---telnet access exists on any port---you would have to block access to that specific port (acl).

"If a service is listening on a port will you always be able to telnet to it?"

Not necessarily "listening" on it, per se, but simply if the port is open and/or enabled, then yes---telnet ability always exists. To separate confusion...

Telnetting into a device on its native port (23) allows a remote console session.

Telnetting into a device by redirecting the port to a different port verifies layers 1-7 connectivity---telnet in general tests all 7 layers of the OSI model.

Thursday, August 19, 2010

SVN Pre -commit script example. Deny commit if no comment or it is less than 10 caracters

This script will deny commit if there is no comment or comment is less than 10 characters.




########################################
REPOS="$1"
TXN="$2"
min=10






comment_entered=`svnlook log -t $TXN $REPOS`




comment_lenght=`echo $comment_entered  | wc -c | tr -d ' '`


if [ "$comment_lenght" -lt "$min" ]; then
 echo "" 1>&2
  echo "*** Your commit has been blocked because you did not give any log message or your log message was less than 10 caracters." 1>&2
  echo "Please write a log message describing the purpose of your changes and then try committing again." 1>&2
  exit 1
else
  exit 0
fi

##################################


You can copy paste above content in /svnroot/yourepo/hook/pre-commit . Good thing is that you don;t even need to make any changes in this script. 

Enjoy 

Amit ......


Wednesday, August 18, 2010

SVN post commit Hook script to send mail when there is a commit.

*********
-> Go to repository hook directory like /svn/yourepo/hook
-> now copy post-commit.tmpl to post-commit
-> Now edit post-commit to call another script which will executed after every commit.
 Enter these lines in post-commit file
REPOS="$1"

REV="$2"

nohup /svn/yourrepo/hooks/script.sh $REPOS $REV >> logfile &

Save this file and make a new file with name script.sh

**************************
Now content of this file will be.

********************************************
#!/bin/sh




REPOS="$1"

REV="$2"



echo >/svn/yourrepo/hooks/mailcontent.txt



REPO_URL_TO_MATCH1="path of branch"



AUTHOR=`svnlook -r $REV author $REPOS`

RESULT1=`svnlook dirs-changed "$REPOS" -r $REV
grep -e "$REPO_URL_TO_MATCH1"`;

echo "Result1 is : $RESULT1";



DATE=`date`;



for pattern in $RESULT1

do

if [ ! -z "$pattern" ];

then

echo "Revision $REV committed on $REPOS" > /svn/yourrepo/hooks/mailcontent.txt

echo "Date: $DATE" >> /svn/yourrepo/hooks/mailcontent.txt

echo "Committer: $AUTHOR" >> /svn/yourrepo/hooks/mailcontent.txt

echo "Repository: $REPOS" >> /svn/yourrepo/hooks/mailcontent.txt

echo "COMMIT MESSAGE:" >> /svn/yourrepo/hooks/mailcontent.txt

echo "----------------" >> /svn/yourrepo/hooks/mailcontent.txt

svnlook log -r $REV $REPOS >> /svn/yourrepo/hooks/mailcontent.txt

echo "CHANGED FILES:" >> /svn/yourrepo/hooks/mailcontent.txt

echo "$pattern" >> /svn/yourrepo/hooks/mailcontent.txt

svnlook changed -r $REV $REPOS >> /svn/yourrepo/hooks/mailcontent.txt

mail -s "New Commit - $REPOS" mail_id@example.com -- -rsendermailid@example.com < /svn/yourrepo/hooks/mailcontent.txt

#rm -rf $MSG

fi

done

*******************************************
TAG
 
Example Hook script
 
-> Hook script to send mail on commit
-> SVN commit hook

Using mail command in Linux

This is actaully very simple. Just adding the something which is difficult to find.

How to give sender (From) part in Linux mail command.

mail -s "Subject content" amitgupta0105@gmail.com -- -rfrom@mail.id
 < mail text file.

In above keep in mind the from it has be like (-- -r) after than immediate sender mail id. Don;t get confuse by r coming in front of from@mail.id .


TAG :
 give sender mail ID in linux mail commad
From mail ID in linux mail command.

Friday, August 13, 2010

Create encrypted loopback filesystems on Linux

Using the loopback filesystem interface, you can create encrypted filesystems very easily. These filesystems are great for storing sensitive documents be it SSH or GnuPG keys, financial documents, etc.


To begin, you'll need to load the aes and cryptoloop modules in the kernel if they are not already available. This can be done by executing:

# modprobe cryptoloop


# modprobe aes



Most modern Linux distributions provide these modules from the get-go, so you shouldn't have to recompile the kernel. Once this is done, create the filesystem container, associate it to a loopback device interface, and format it:



# dd if=/dev/urandom of=enc.img bs=1M count=50



# losetup -e aes /dev/loop0 enc.img



Password:



# mkfs -t ext2 /dev/loop0



# mount /dev/loop0



# mount -o loop,encryption=aes enc.img /media/disk



Very -2 Important is that this will work for once, but when you will reboot what you need to do.

you need to just give this command again.
# mount -o loop,encryption=aes enc.img /media/disk


but the problem is this command will fail. So make a file /etc/rc.modules (if it is already not there) and copy this contents in this file.

modprobe aes
modprobe loop
modprobe cryptoloop

(this will load modules during boot time so will not get the module error). Else you have to run modprobe command after every reboot.




(Waise yahan tak ka hi kaam ka hai baki sab waise hi hai )




The first step creates an empty image file called enc.img with a size of 50 MB; you can increase this by changing the count value. Next, use losetup to associate the enc.img file to the /dev/loop0 device and tell it that the device is to be encrypted with AES encryption. This command uses 128-bit AES encryption; look at the losetup manpage to see what other encryption types you can use. You will have to provide a password that will be used from that point forward to access the image.



Next, the filesystem is formatted with the ext2 filesystem. Finally, it is mounted to /media/disk. The options passed to mount tell it to use the loopback interface and the encryption type needed. When you call mount, you will have to provide the password you used to encrypt the image.



Putting this kind of image in /etc/fstab will not work unless you want to be prompted for your password on each boot. Instead, this should be accessed as needed. For instance, you could store the file as ~/.enc.img so it's hidden from normal view, with mode 0600 permissions. Wrapper scripts could be written to mount and umount the image easily:



#!/bin/sh



# mount ~/.enc.img



mkdir -p /media/secure && mount -o loop,encryption=aes ~/.enc.img /media/secure



And to unmount the volume when you're finished with it:



#!/bin/sh



# umount /media/secure



umount /media/secure && rmdir /media/secure



These two commands could be saved as ~/bin/ms and ~/bin/ums respectively. Alternatively, you could add the following to ~/.bashrc and uses aliases instead:



alias ms="mkdir -p /media/secure && mount -o loop,encryption=aes ~/.enc.img /media/secure"



alias ums="umount /media/secure && rmdir /media/secure"



Using encrypted filesystems for on Linux is extremely easy and sensible, especially for laptops or when dealing with very sensitive files.

Tuesday, August 10, 2010

save file permission

saveState.pl






Code:

my $tree = $ARGV[0];



foreach $file (`/usr/bin/find ${tree}`)

{

chomp($file);

(undef, $inode, $mode, undef, $uid, $gid, $undef, $size, undef, undef, undef, undef, undef) = stat($file);

$permissions = $mode & 07777;

printf "%i:%i:%04o:%i:%i:%s\n", $inode, $size, $permissions, $uid, $gid, $file;

}

applyState.pl





Code:

#Does not save ACL...



my $stateFile = $ARGV[0];



open(STATE_IN, "<$stateFile");



foreach $line ()

{

chomp($line);

my ($inode, $size, $permissions, $uid, $gid, $file) = split(':', $line, 6);

if( -f $file )

{

#add inode / size check here if desired

chmod(oct($permissions), $file);

chown($uid, $gid, $file);

}

}



close(STATE_IN);

########################## Example #################





Code:

9:45am root@hyperion /research/src/state_saver #>perl saveState.pl /research/src/state_saver/ > test



9:45am root@hyperion /research/src/state_saver #>cat test

1846211:512:0755:0:0:/research/src/state_saver/

1846214:0:0644:0:0:/research/src/state_saver/test

1846213:365:0644:0:0:/research/src/state_saver/applyState.pl

1846212:332:0644:0:0:/research/src/state_saver/saveState.pl



9:45am root@hyperion /research/src/state_saver #>l

total 10

1846211 drwxr-xr-x 2 root root 512 Jan 13 09:45 .

1846210 drwxr-xr-x 3 root root 512 Jan 13 09:30 ..

1846213 -rw-r--r-- 1 root root 365 Jan 13 09:44 applyState.pl

1846212 -rw-r--r-- 1 root root 332 Jan 13 09:44 saveState.pl

1846214 -rw-r--r-- 1 root root 219 Jan 13 09:45 test



9:45am root@hyperion /research/src/state_saver #>chmod 777 saveState.pl



9:45am root@hyperion /research/src/state_saver #>l

total 10

1846211 drwxr-xr-x 2 root root 512 Jan 13 09:45 .

1846210 drwxr-xr-x 3 root root 512 Jan 13 09:30 ..

1846213 -rw-r--r-- 1 root root 365 Jan 13 09:44 applyState.pl

1846212 -rwxrwxrwx 1 root root 332 Jan 13 09:44 saveState.pl

1846214 -rw-r--r-- 1 root root 219 Jan 13 09:45 test



9:45am root@hyperion /research/src/state_saver #>perl applyState.pl test



9:46am root@hyperion /research/src/state_saver #>l

total 10

1846211 drwxr-xr-x 2 root root 512 Jan 13 09:45 .

1846210 drwxr-xr-x 3 root root 512 Jan 13 09:30 ..

1846213 -rw-r--r-- 1 root root 363 Jan 13 09:46 applyState.pl

1846212 -rw-r--r-- 1 root root 332 Jan 13 09:44 saveState.pl

1846214 -rw-r--r-- 1 root root 219 Jan 13 09:45 test



9:46am root@hyperion /research/src/state_saver #>
 
 
 
 
--------------------------------------
http://www.unix.com/shell-programming-scripting/42750-shell-script-save-restore-files-dir-permissions.html
_____________________________

Wednesday, August 4, 2010

Text searching in linux

This command would search for "a string" and return 3 lines before and 4 lines after. To give it a test create a file called testfile.txt and put a bunch of random stuff in it and on a few lines put "a string" without the quotes.




grep -B3 -A4 "a string" testfile.txt >grepout.log



This would output the results to grepout.log, but you can remove the >grepout.log part to have it right on the screen. Outputting to a file is not a bad idea when searching huge amounts of files since you will probably get lot of matching stuff so afterwards it will be easier to sort through.



Here's how you'd do the same thing, but instead of searching testfile.txt you want to search all the files in /etc.



grep -R -B3 -A4 "a string" /etc/*



This is surprisingly pretty quick too, here's some of what it got on my system. Just part of what appears to be a configuration file. From this point on I know that "a string" is in that file so I can vim to it and do a search (using /a string in the vim command line) and voila, found a string, without having to manually check in each file. Funny how it's talking about regular expressions, since we'll be looking at those later.

Grep with commands


Grep is not only limited to searching for files, you can also limit command output to a specified string. For example



locate spam
grep ^/etc/

Grep with commands


Grep is not only limited to searching for files, you can also limit command output to a specified string. For example



locate spam
grep ^/etc/


The locate command is used to locate files on the hard drive. So typically locate spam would list every single file that matches spam, but that's a huge list, so using a command such as the one above will limit to /etc folder. The ^ tells it that it has to start with that. So /data/etc/spam would not match. Without the ^, it would match.



You can use grep with dir as well. If you're looking for the hosts file but don't remember if it has an extension, or if it has an s then you could use this command:



dir
grep "host"



Returns:

host.conf pam.d vsftpd.conf

hosts pam_smb.conf vsftpd.ftpusers

hosts.allow pango vsftpd.user_list

hosts.deny paper.config warnquota.conf
 
Remember, it goes by line, so this is why pam.d and vsftpd.conf and other files are listed, as long as "host" is somewhere on that line.




Another command you can use grep with is top. Top is used to show system usage (sort of like the task manager in windows) and it updates every few seconds.



So issuing top
grep "httpd" would execute top, but only show the lines which httpd is in, so every few seconds, a new line would be printed, if, and only if, httpd is in the top section of cpu usage. This is especially good if you are monitoring a program that is using too much resources since every few seconds it will print a new line with the usage info. And if it stops printing, then you know the program ceased using up enough cpu to apear in the top list.



Grep can be used in conjunction with any command, as far as I know and it has many features not mentioned so far so the next page will simply show some examples of advanced uses of grep
 
 
http://www.iceteks.com/articles.php/grep/1
 
 
 
 
Linux: Find text in a large number of files


retweetIf you need to find a string in a file, you would typically use:

grep -H "string to find" filename.ext

However, grep doesn’t handle a large number of files well. If you specify grep "string" * or even grep "string" `find ./`you may find yourself facing this error:

bash: /bin/grep: Argument list too long

If you need to search for a string in a lot of files then you can use a simple bash script to do the searching for you.



In this sample, I am looking for a string “sample string” in a directory named “./sample/”:

for i in `find ./sample/`; do grep -H "sample string" $i; done

This uses the find command to do the searching. It actually returns a list of filenames, which we can then grep one-by-one. The -H option tells grep to let us know the filename it found the string in so we can go right into that file to find the location of it.



So if you are a normal user and that you are facing the "missing files" problem in Linux, don't worry, I will show you the most common methods in solving this issue:




Find files that contain a text string



grep -lir "text to find" *



The -l switch outputs only the names of files in which the text occurs (instead of each line containing the text), the -i switch ignores the case, and the -r descends into subdirectories.



Find files containing search terms on Ubuntu



To find files containing keywords, linux has a powerful command called grep, which you can use to find the lines inside any file or a list of files.



grep -i -n 'text to search' *



List files containing text



Used to recursively search a directory for files containing a string, output the names of the files and the line number. This will search all regular files in for.



grep --with-filename --line-number `find -type f`

Brain Excersises

http://www.brainmetrix.com/
http://www.brainarena.com/
http://www.braingle.com/

Thursday, July 29, 2010

Check linux kernel and CPU is 32 bit or 64 bit. | How many CPU in your server.

Find Out If Running Kernel Is 32 Or 64 Bit

$ uname -a
Linux hostname 2.6.9-89.0.20.ELsmp #1 SMP Tue Feb 2 18:35:37 EST 2010 i686 i686 i386 GNU/Linux

as there is i386 in end this means Kernel running is 32 bit. Still you can have 64 bit processor to check the same you need to see /proc/cpuinfo

$ grep flags /proc/cpuinfo

flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl cid xtpr


flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl cid xtpr

- If lm is present that means CPU is 64 bit if it is not there than 32 bit. We hardly going to see 16 bit proccessor.

■lm means Long mode - 64 bit CPU

■Real mode 16 bit CPU
■Protected Mode is 32-bit CPU
under "flags" parameter u will see various values.


Among them . u will find one of them with name "tm(transparent mode)" or
"rm(real mode)" or "lm(long mode)"
1. rm tells ,it is a 16 bit processor
2. tm tells, it is a 32 bit processor
3. lm tells, it is a 64 bit processor
 
You can also try this to find out if your CPU/Processor is 32 bit or 64 bit:

# getconf LONG_BIT
32

Linux display CPU information

Now to get the number of CPU in your Linux server.

Need to give command

#cat /proc/cpuinfo

  •  If you have two CPUs in cpuinfo with the same physical id and core id, than – it is hyperthreading!  (Hyperthreading is when One CPU is seen as 2 Logical CPU by OS, using Hyperthreading OS can send 2 instruction for processing to CPU)
  • When you have all CPUs with different physical ids, than you have SMP,
  • when you have CPUs with one physical id and different core ids – you have one multicore CPU. And so on…





 

How do I delete all but one directory in Linux

$ shopt -s extglob


$ rm -rf !(savedir)



you Korn shell users can ommit the first line.