Dynamic RSS generation using SPARQL and Yahoo Pipes

This demo shows how to create an RSS 2.0 feed for RDF data using SPARQL and Yahoo! Pipes.
Following is a step-by-step explanation on the workflow of this demo.
Step 1 locate RDF data (TWC LOGD in the News)
The RDF data used in this demo is collected by parsing RDFa metadata embeded on web pages on http://data-gov.tw.rpi.edu, and here is a link to an example page. The RDFa annotation of the example page can be extracted by any RDFa parser, and here is a link to the parse result.
Step 2 compose a SPARQL query to prepare the content of RSS
The following SPARQL query (using SELECT construct) is used in this demo. The variable names are carefully selected to match the terms used in RSS. Therefore, users should keep using the same variable names(including title, image, description, link, image, and pubDate) when they want to compose RSS content on the other RDF data.
PREFIX rss: 
PREFIX dcterms: 
PREFIX foaf: 

 GRAPH ?g {
 _:x a 
    ;foaf:name ?title
    ;foaf:depiction ?image
    ;dcterms:description  ?description
    ;dcterms:source ?link
    ;dcterms:created ?pubDate

This SPARQL also have some limitations as users cannot control the tile and description of the RSS channel. Although "CONSTRUCT" primitive looks promising in solving this issue, it also has limitations. First, SPARQL results uses XSD:dateTime to encode date (e.g. "2010-05-21T00:00:00") while RSS 2.0 requires data formated in RFC822 (e.g."Thu, 20 May 2010 17:00:00 -0700"). Second, the output RDF/XML syntax used "rdf:Resource" as element tag, while RSS 2.0 demands specific XML format (they are looking for "<channel>" and "<item>" instead). Therefore, we stick to "SELECT" based data construction and generate RSS using a web service-- Yahoo! pipes in this demo.
Step 3 query RDF data in a SPARQL endpoint using SparqlProxy and then get JSON back
The data in this demo are all loaded at a triple store with SPARQL endpoint  http://data-gov.tw.rpi.edu/ws/lodc.php. For each page, it RDF metadata is stored in one named graph in the triple store. The SparqlProxy query interface and choices of parameters can be viewed via this link.
  •  check "Query URI" and paste Query URI after
  •  check "sparqljosn" as output format * click "query" button

The query results can be accessed by application via this link and viewed in text format via another link.  
Step 4
build a Yahoo Pipes! web service that convert the SPARQL query results (in SPARQL/JOSN format) into an RSS2.0 data.the code is available at http://pipes.yahoo.com/pipes/pipe.info?_id=5d8842906152a7d3152a128666970f13 . Following are some notable points:
  • you need a yahoo ID to access Yahoo Pipes!
  • SPARQL/JOSN is used to provide access to data structure. Yahoo! Pipes has hard time deal with the XML version
  • representation of date is not compatible between RDF (ISO  and RSS (required RFC822)
Finally, the result RSS is generated, and Yahoo! also provided some human readable page for the RSS feed (wiht images). The RSS feed is also imported by TWC LOGD front page's RSS aggregator.
Uses Technology: 
Uses Technology: 
Uses Technology: 
No votes yet

Warning: Table './drupal/watchdog' is marked as crashed and last (automatic?) repair failed query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (0, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:374:\"Table &#039;./drupal/accesslog&#039; is marked as crashed and last (automatic?) repair failed\nquery: INSERT INTO accesslog (title, path, url, hostname, uid, sid, timer, timestamp) values(&#039;Dynamic RSS generation using SPARQL and Yahoo Pipes&#039;, &#039;node/3896&#039;, &#039;&#039;, &#039;;, 0, &#039;0iudkuebi6o3iht71m1a9fee70&#039;, 35, 1618679889)\";s:5:\"%file\";s:58:\"/data/www/html/drupal/modules/statistics/statistics.module\";s:5:\"%line\";i:63;}', 3, '', 'https://logd.tw.rpi.edu/dem in /data/www/html/drupal/includes/database.mysqli.inc on line 134