Customize RSS News Feeds With Yahoo Pipes
Yahoo Pipes is a very sophisticated online tool which might look intimidating at first glance due to its technical nature; it is however not as complicated to use as you can create great services and tools with minimal user input.
One major use of Yahoo Pipes is the customization of RSS News Feeds. It does not really matter if one RSS News Feed or multiple should be customized, both is possible.
Update: Yahoo has shut down Yahoo Pipes. You may be interested in the third-party project Pipes which replicates some of the functionality that Yahoo Pipes offered.
The only requirement for using Yahoo Pipes is a Yahoo account which can be obtained freely if needed. I'm going to walk you through the customization of the Softpedia Windows RSS News Feed which lists freeware, demos, adware, trial and shareware software programs in its feed. Since I'm only interested in the free software I decided to use Yahoo Pipes to filter out all the other items.
You can use the same technique for your favorite website. Say you own a Windows PC and are subscribed to the Lifehacker RSS News Feed. You might want to filter out any item that describes software for an Apple Macintosh computer.
The Yahoo Pipes interface is divided into three panes. The left side contains the different elements that can be dragged and dropped on the main canvas area. You can use them to load RSS News Feeds and filter data among many other things. The last pane is the debugger area which displays the data of the selected element and other information.
We want to fetch an RSS News Feed and drag the Fetch Feed element into the canvas area. This element requires at least one RSS News Feed url so copy and paste on into the box. If you click on that box you will notice that the debugger has loaded the titles of the RSS News Feed at the bottom. You can click on the titles to see variables like descriptions or titles of the items.
You can click on the + icon next to URL to display another form to paste another RSS News Feed URL.
Click on the Operators entry in the left menu and drag the Filter element to the main window. Connect the Fetch Feed and Filter elements by pressing the mouse button over one of the nodes and (while holding the mouse button) moving it to the other node.
The elements are now connected. The process begins by fetching the RSS News Feed, sending it to the filters where they are customized and then to the output which is already available at the end. All that remains to be done is to create the filters.
You can block or permit items that match the filters. To do that you need to find differences or similarities. Softpedia uses classifications in their item titles. To find out about them you can head to the debugger area and click on a RSS News Feed item. You will notice that there is an entry called y:title which contains the title of the news item. You find the classification at the end of each title, for instance freeware, adware or shareware.
I decided to block all unwanted classifications from the news feed by selecting item.y:title > Contains > Classification, for example item.y:title > contains > Shareware. This will block all items with the Shareware classification in the title. I create three additional rules for Trial, Demo and Adware.
It would have been possible to do it the other way round. To allow only RSS News Feeds with titles like freeware or gpl. A click on the Filter element should update the debugger. It should only display freeware and gpl news items. Now connect the Filter element with the Pipe Output element.
The Pipe Output Element should display the same content in the debugger as the Filter element because nothing is changed in between.
A click on Save saves the pipe and enables a Run Pipe option in the header area. A click on that opens the Pipe profile with a preview area and options to use it as an RSS Feed or embed it on websites.
Is resulting feed real time or there is lag between update of original feed and custom feed?
Rarst I let you know, currently checking it out and waiting for new items to be published ;)
Most items are delayed which seems to suggest that Yahoo is only updating the feed every x minutes. The largest difference that I spotted was 30 minutes while the closest was -1 ;)
Tried it out (also with softpedia, 200+ items a day from them is bit too much :) ).
Really solid and seems to work nice.
Didn’t work in Opera for me (one of those rare over-ajaxed sites I guess).
That was exactly my reasoning for looking at Yahoo Pipes.