rel is a  link attribute and setting its value to no follow tells search engines that certain links on your webpage should not be followed or more importantly should not pass your web page’s hard earned reputation.

<a href=”www.test.com” rel=”nofollow”>test</a>

This can generally be very useful when you have a blog or other areas where you have user generated content like forums, guest books etc.

Suppose you have a blog with public commenting turned on, now links within those comment may pass some your pages reputation to other pages that you may not be willing to endorse. Specially blog commenting is very susceptible to such kind of spammy links. And linking to sites that google considers spammy can affect your website big time. However you can keep a check on such kind of links by CAPTCHA and turning on comment moderation.

Another advantage adding no follow links is suppose you are writing a content and want to add some kind of reference to a particular website but at the same time do not want to pass on your pages reputation you can use this property. For Example: Suppose you are writing a blog post of spammers and want to refer to his website and warn fellow bloggers by putting his links on your post, but you certainly don’t want to share your page’s value. So just by adding rel=”nofolow” to the links will solve your problem.

Also if you want that none of the links on your webpage should be followed you can add that in your robots meta tag in the head section of your website.

<meta name=”robots” content=”nofollow”>

This will nofollow all the link on that webpage

Note: some of the points written above are taken from Google’s Search Engine Optimization Guide