Warning: Table './drupal/watchdog' is marked as crashed and last (automatic?) repair failed query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (0, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:7:\"warning\";s:8:\"%message\";s:40:\"Creating default object from empty value\";s:5:\"%file\";s:57:\"/data/www/html/drupal/modules/taxonomy/taxonomy.pages.inc\";s:5:\"%line\";i:33;}', 3, '', 'https://logd.tw.rpi.edu/category/keywords/datagov', '', '10.0.1.254', 1575578435) in /data/www/html/drupal/includes/database.mysqli.inc on line 134
Home

data.gov

Data.gov Catalog Search

Description: 
This demo provides a search and faceted browsing interface for all data.gov datasets from the raw data catalog, the tool catalog and the geodata catalog.
Contributor:
Contributor:
Live Demo: 
keyword:
Uses Dataset: 
Uses Dataset: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses SPARQL Query: 
Uses SPARQL Endpoint: 
Screen Shot: 
Thumbnail: 
  • Login to post comments
  • 4359 reads

Data.gov survey: dataset modification dates compared

Description: 
This demonstration shows when data.gov datasets were last updated, as determined by their HTTP responses. How up-to-date are the datasets listed in data.gov? We start with a time line histogram for a moment in time.
Contributor:
On the afternoons of __ and __, we crawled data.gov's links to datasets and recorded the modification dates reported by the source agency's HTTP servers. Of __ data.gov datasets, __ datasets returned modification dates on __ and __ returned modification dates on __. The list of data.gov datasets came from the 13 Sept 2010 version of their Dataset 92.
change blue crawl: change red crawl:

Number of Datasets Modified since __

Process used

The following diagram shows the process used to obtain the last-modified dates of the data.gov datasets.

Use of provenance

The following diagram illustrates how the Proof Markup Language was used to record the HTTP headers obtained from the government agency servers. A portion of this data graph is selected by the SPARQL query listed above (the headers are excluded). The red links show the HTTP redirects and HTML anchor hrefs from the data.gov details page to the actual dataset URL on the hosting agency's domain. The green boxes represent URIs for the information obtained (HTTP headers), while the purple boxes associate the information received with its source and when it was received. For more discussion on how provenance is used in LOGD and csv2rdf4lod, see A look at how csv2rdf4lod incorporates provenance into its tabular conversions.
Uses Dataset: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses SPARQL Endpoint: 
Screen Shot: 
Thumbnail: 

Data.gov survey: Raw dataset modification dates

Description: 
This demonstration shows when the data.gov datasets were last updated, as determined by their HTTP header responses. How up-to-date are the datasets listed in data.gov? We start with a time line histogram for a moment in time.
Contributor:
Live Demo: 
Try the new version to compare crawls.
On the afternoon of 21 Sept, 2010, we crawled data.gov's links to data sources and recorded the modification dates reported by the source agency's HTTP servers. Of __ data.gov datasets, __ datasets returned HTTP last-modified dates. The list of data.gov datasets came from the 13 Sept 2010 version of their Dataset 92.

Process used

The following diagram shows the process used to obtain the last-modified dates of the data.gov datasets.

Use of provenance

The following diagram illustrates how the Proof Markup Language was used to record the HTTP headers obtained from the government agency servers. A portion of this data graph is selected by the SPARQL query listed above (the headers are excluded). The red links show the HTTP redirects and HTML anchor hrefs from the data.gov details page to the actual dataset URL on the hosting agency's domain. The green boxes represent URIs for the information obtained (HTTP headers), while the purple boxes associate the information received with its source and when it was received. For more discussion on how provenance is used in LOGD and csv2rdf4lod, see A look at how csv2rdf4lod incorporates provenance into its tabular conversions.
Uses Dataset: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses SPARQL Query: 
Uses SPARQL Endpoint: 
Screen Shot: 
Thumbnail: 
  • Login to post comments
  • 5478 reads

US Government Agencies' Contributions at Data.gov

Description: 
This demo shows how many raw datasets have been contributed to Data.gov by each US government agency.
Contributor:
Contributor:
Live Demo: 
Interesting Observations This demo queries the data.gov catalog to list all the agencies that submitted data to data.gov, then counts the number of datasets each submitted. From the visualization, web users are presented with the following observations:

Technical Highlights

In this demo, SPARQL query results are displayed using various Google Visulization APIs including pie chart, bar chart, line chart and table.
This demo also show the use of aggregation function in SPARQL (we used virtuoso syntax) 
# NOTE:
#  this query used SPARQL 1.1 feature, and its SELECT clause is customized to virtuoso syntax
#  we only query raw data catalog
#
PREFIX dgp92:  <http://data-gov.tw.rpi.edu/vocab/p/92/>
SELECT  ?agency , count(*) AS ?cnt
FROM NAMED <http://data-gov.tw.rpi.edu/raw/92/data-92.rdf>
WHERE { 
GRAPH <http://data-gov.tw.rpi.edu/raw/92/data-92.rdf>
{
 ?entry dgp92:agency ?agency. 
 ?entry dgp92:data_gov_data_category_type "Raw Data Catalog" .
}
}
GROUP BY ?agency
ORDER BY DESC (?cnt )
Uses Dataset: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses Technology: 
Uses SPARQL Query: 
Uses SPARQL Endpoint: 
Screen Shot: 
Thumbnail: 
  • Login to post comments
  • 1500 reads
Syndicate content

Warning: Table './drupal/watchdog' is marked as crashed and last (automatic?) repair failed query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (0, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:335:\"Table &#039;./drupal/accesslog&#039; is marked as crashed and last (automatic?) repair failed\nquery: INSERT INTO accesslog (title, path, url, hostname, uid, sid, timer, timestamp) values(&#039;data.gov&#039;, &#039;taxonomy/term/23&#039;, &#039;&#039;, &#039;10.0.1.254&#039;, 0, &#039;09o27u5p5mmhj1cef6pqsoaau1&#039;, 84, 1575578436)\";s:5:\"%file\";s:58:\"/data/www/html/drupal/modules/statistics/statistics.module\";s:5:\"%line\";i:63;}', 3, '', 'https://logd.tw.rpi.edu/category/keywords/datagov', '� in /data/www/html/drupal/includes/database.mysqli.inc on line 134