RSS, Atom, and XML...

[gonzo]

I'm looking for info on cutting edge usage of RSS/XML...specifically if it is possible to use an RSS feed to modify a 3rd party website that has been coded to recieve the RSS feed internally.

Any ideas, programming dudes?

[sfotex]

By nature RSS would typically be set up to 'modify' the content of a web page/site that is set up to get a RSS feed, so the broadcaster would send out an update for the client that would display the update on a web page...what are you trying to do?

[fez]

yeah, absolutely, it would be pretty easy to parse out the xml and use it to modify another website. depending on what you mean by 3rd party however, you probbaly cant use it to modify someone elses website unless you wanted to just use custom javascript to modify it client side.

[gonzo]

Thanks dudes...i am trying to figure out if a client site could subscribe to an RSS feed within the code of their site and then have the rss feed then periodically change the code of the client's site. My experience with rss is limited to what i see on my browser, etc. From what you guys are saying though it sounds like it would be easy to subscribe to an rss feed within a site's code to change something like news headlines or something, so it would not be hard to change where the subscription data was sent within the site?

Any idea where i can find reading on this? Or what sort of code/software you need to imbed an outside RSS feed within a site's code?

[fez]

an rss feed is nothing but xml. and it is referenced by url. All it takes to add a rss feed to a site is a link tag pointing to the rss xml file.

<link rel="alternate" type="application/rss+xml" title="RSS" href="Rss.aspx" />

like that. That references the Woot.com rss feed. so if you go to:
http://www.woot.com/Blog/Rss.aspx

you should be able to see the raw xml of the woot.com rss feed. So if you wanted to do something on your website about say what woot.com is selling at the current time, you could just parse whatever info you needed out of the woot rss xml.
In general with rss, the most recent entry would be first (there is also a publication timestamp too). so you would just look for the <items> tag and grab the first <title> and <description> if you needed it.

How you go about this is up to you, your preferences, and your abilities.
You could do this client side or server side.
Theoretically, you could use JavaScript to do it client side. but if you have a site that generates any kind of traffic, you will be getting the xml doc from woot every time someone visits your site. not really too cool.
It would make more sense to do it server side. this way you can use php, perl, ruby, python, c/c++/c#, basically any cgi capable language. Plus instead of getting the rss xml document everytime you use it, you can cache it locally and run a script to check to see if it has changed. and if it has replace the local one.

clear as mud?

[sfotex]

There's a bunch of api's (remember those ) out there to work with rss news feeds server side.

[gonzo]

you guys rule.

what do you mean by "server side"? on the client's server?

[sfotex]

server-side i.e. the server/computer that is creating the web pages that are sent to the person (or client) requesting them. You can create the web page that includes the RSS content on the server side, and send it to the whole web page to the client, or you can write a program in javascript that you include in the web page, so when the user's browser recieves the web page, it runs the javascript program. The program will run, get the RSS newsfeed, and display the RSS info. (Fez, would you have problems with cross scripting security if the feed is from a different server?)

[fez]

I wouldnt think there are any problems with cross-site scripting with rss just from reading the xml file. I would treat all data parsed out of the xml file as tainted though and filter the html, probably scrubbing image and script tags. Definitely filter against sql injection if you are going to database any info from the xml.

<edit>what i should have said is 'yes, cross site scripting would be an issue, here's how you should deal with it...'. </edit>

I hadn't realized there were rss specific apis. I was just thinking of using apis for reading the xml.

[gonzo]

Thanks again for your help guys.

I found list list of RSS parsing programs:

http://www.rssgov.com/rssparsers.html

Correct me if I am wrong but, to do this sort of thing "server side," one would find one of these programs that was written in the same langauge as the site and insert it into the site's file? But this would only work with a dynamic site? But most commercial high traffic sites are dynamic now anyways?

[gonzo]

Sfotex: are the programs above the type of server side APIs you had in mind when you made your earlier post?

I clicked around on the site I posted above and looks like these will do the trick.

This:

http://www.jciti.com/news/index.php

is a site that is generated entirely from RSS feeds but the title of the feed is see in the actual code of the site, not as a javascript...So, I'm tempted to think you could put something like this anywhere in the code of your site...

[sfotex]

Gonzo,
It looks like there's alot of programs on the link you posted that you can install on your site, setup, and then reference via javascript. I think this would be the easiest way to get RSS going for you. So you would install a program, and in the program setup you might specify the name of the RSS feed, how often to check for a refresh, etc. It would probably spit out some javascript for you that you would paste into the web page where you want to display the RSS info. The JS would point to the program that you just setup.


FWIW, most, if not all, large commercial web sites are Dynamic sites. When you go to egay for example and type 'www.egay.com' into the browser, you are firing off a request to the egay server to run a program on their server. This program makes some calls to a database, does some logic on the info it gets back - i.e. is this user's account have any over do payments - if 'yes' then include some text on the page that says 'Payup sucker' . The program builds a complete web page in html and hands it back to your browser to look at. Of course there are exceptions to this model - plugins (shockwave, java applet, etc) and AJAX can call back to the server after a web page is loaded to get new info...

[fez]

Sfotex: are the programs above the type of server side APIs you had in mind when you made your earlier post?

I clicked around on the site I posted above and looks like these will do the trick.

This:

http://www.jciti.com/news/index.php

is a site that is generated entirely from RSS feeds but the title of the feed is see in the actual code of the site, not as a javascript...So, I'm tempted to think you could put something like this anywhere in the code of your site...

Howdy Gonzo,

Since the url you show to the aggregator site ends in .php, and looking at their javascript it looks like nothing that could accomplish building the content of the site, i would assume it is built in server side php. So yes, if you had a php include file that handled parsing out your rss file, you could include it into any php script, reference it in the code and you could generate a page like that pretty easy.

After talking with you yesterday I think i have a pretty good idea on what you are trying to accomplish. One thing I have been thinking about is whether or not you even need to use RSS. RSS is simply xml that is in a specific format. Really for what you are trying to accomplish, you would probably be better off developing your own xml format that more specifically fits your needs. XML tends to be very verbose in describing its data, meaning that a xml file will usually have far more bytes used to describe the data than it has in actual data. If you skipped some of the overhead of generating valid RSS and just worried about the xml you need, your xml file size would likely decrease. This would save you bandwidth and processing speed.

Also, since you have rather specific needs, you probably dont need all the overhead of a general rss parsing program. For instance, most of those programs can handle RSS .9x, 1.0 and 2.0. Since you will be generating your rss yourself, you wouldnt need to handle all the versions. Most programming languages have some sort of XML parsing library that will let you rather easily parse or write any xml document.

for the programs you linked to, you should make sure you check out what type of license they are offered under and think about how that would work with your company.

[gonzo]

Thanks again gents.

sfotex: good to know that most are dynamic - the sites in question are high end retail sites so i am guessing most of them would be dynamic as well.

fez: i smell what you're pooping re: RSS being very verbose - i just need words to show up in the code server side - no links or anything else. it would be great to ditch the extra stuff. i never thought of that because i assumed you needed "RSS" to be able to send stuff from one website to another via feed. at what point and where is that ability created and where could it be lost? is the deal that you can always feed XML?

edit: i guess the question is at what point does a feed become a feed?

[fez]

all you are doing setting up a 'feed' is making a file available on a network. how available it is is up to you. you can make it widely available to the public or you can require credentials of some sort to access it.
the only advantage that rss has is the format. all rss files have to be in a specific format, rss readers know this and know what to do with the data in the rss file. for example, if you have firefox, and a website puts a link tag in its html to an rss formated xml file, then firefox can create a live bookmark. rss makes sense if you can only control one end of the pipe.
since in your situation, you control both ends of the pipe (producer and consumer of the xml) then you can use whatever xml format you want. Most likely a good programmer could make a much more useful xml format for you. also in your case, since the data in the xml seems somewhat propreitary, you should probably require some sort of authentication so that competitors can't see what you are sending but mostly to keep your various customers from seeing each other's data.

I have a few other ideas about this, pm me your email and I will send you an email this evening about it.

[sfotex]

If you need security, etc. and info targeted to specific clients you might want to go with webservices...

Feel free to PM me too if you have any specific questions - i'm more a java server side guy if that helps, dunno what enviroment your running in...