GrabNews

GrabNews is ASP code which will 'grab' RDF/RSS/Atom stndicated feeds from the web and display the news items on a web page. It uses XSLT to transform the newsfeeds into HTML and stores the raw RDF/RSS in a database. It is set to 'grab' the newsfeeds every hour. It will also only display the first ten, but this can be changed in the XSLT stylesheet, news stories.

Of course all of this can be changed to the way you want if you look at the code. If you make any improvements to the code then let me know and send it back to me and I may integrate it into the main code.

Sites to get RSS/RDF/Atom feeds:
Syndic8 - http://www.syndic8.com
NewsIsFree - http://www.newsisfree.com
SNewp - http://snewp.com
Moreover - http://www.moreover.com
Fyuze - http://fyuze.com
Atom Wiki - http://www.intertwingly.net/wiki/pie/FrontPage

REQUIRES
Internet Explorer 5.01 or higher
MSXML3 or higher - http://msdn.microsoft.com/xml
MySQL, MS SQL database or Access

The files you should have are:

news.asp - grabs and displays newsfeeds
addnews.asp - simple form to add newsfeed URLs into the database
news.xsl - XSLT stylesheet to transform syndicated feeds into HTML
icon_speaker.png - icon to indicate podcast availability

newsfeed folder - contains MySQL and Access databases which holds the information for the newsfeeds

To use either the MySQL database you will need to set-up a DSN on your server.
The Access database can either be accessed via DSN or as a local file.

You may need to edit the files news.asp and addnews.asp to set-up your particular database settings. The relevant sections are commented in the ASP files.

If you wish to modify and / or alter this code then please let me know and e-mail me a copy of any modified code to
dazzle@edazzle.net


DATABASE
I use MySQL and have included the MySQL database which I use.
If you don't use MySQL then the database details are described below:

DSN: cm
Database Name: newsfeeds
Table Name: sites

Columns:
site VARCHAR(20) - this is the name of the site e.g. Slashdot
url VARCHAR(100) - URL of the RDF/RSS newsfeed e.g. http://slashdot.org/slashdot.rdf
time INT(2) - hour when the newsfeed was last grabbed, this will be automatically updated when the newsfeeds are grabbed or if you use the addnews form it will automatically be added in. Can be NULL.
xml text - this holds the raw XML from the newsfeed, will be automatically updated when the newsfeed is grabbed or when a new site is added using the addnews form
ID int(11) - PRIMARY_KEY and should be auto_increment

MySQL is a cross-platform, open source database available from http://www.mysql.com

PROXYCFG AND ServerXMLHTTP

GrabNews now uses ServerXMLHTTP rather than XMLHTTP. Using ServerXMLHTTP requires configuring proxy information on your WinNT server. To do this you will need the proxy configuration utility available from:
http://msdn.microsoft.com/code/default.asp?url=/code/sample.asp?url=/msdn-files/027/001/468/msdncompositedoc.xml

To configure your proxy information follow these instructions:
First, let's assume that a proxy server named "myproxy" exists on your local network that must be used to route HTTP requests to the Internet. So you would typically run proxycfg.exe like so:

proxycfg -d -p myproxy "<local>"

If you do not need a proxy server to access the Internet, then just run:

proxycfg -d

ServerXMLHTTP is only available on the WinNT version of MSXML and you must be using at least MSXML3 and Internet Explorer 5.01 or higher
I am using ServerXMLHTTP because it doesn't utilise URL caching which was causing problems when fetching newsfeeds updates.


ERRORS
If you receive the error - 'Invalid character found in file content' then have a look at this web page, http://www.perfectxml.com/msxmlAnswers.asp?Row_ID=64, which explains the problem with encoding of special characters and XML.



TO DO
Add ASP pages for editing the database information
Better error / URL checking

This code is freely available and comes with no warranties or guarantees and is covered by the GNU GPL - http://www.gnu.org

If you use this code on your website then let me know and I will use your website as an example of GrabNews in a live situtation.

As usual any comments, queries, bugs, help etc then contact me at dazzle@edazzle.net

(c) Paul Millar 2001 - 2005