spacer
Home Events News Links People Catalog Admin
spacer

Questions or
comments:
webmaster


Updated:
2006-02-15

Project 2 Instructions

Project 2 gave you over four weeks to coordinate a shared document that responded to the requirements below. I gave you a solid chunk of time because I thought it might be a new academic experience for you to participate with seven people in a group. To be sure, with Web-based technologies and geographically-distributed teams, documents are generated often by larger groups that don't have the luxury of communicating in person. Of course, I encouraged you to meet in person if you could coordinate such meetings. But perhaps, in the spirit of the class, I thought you might want to try to do everything electronically. I recommended that you negotiate, elect, or approve an interested coordinator to be responsible for putting together the rough drafts for your project submission to me. I hope you think through how you could be a better group participant the next time a similar situation arises in your life. If you found yourself energized by working on a team of six others and can imagine getting access to some very smart and experienced people to work on a dream project on which you'd like to participate. I can vouch that the experience is exhilirating, especially when the team takes advantage of worldwide time zones. I really like asking a question to the team before bed and having multiple answers in my in-box when I awake because they have worked while I slept (but not pulled an all-nighter because they live where it is midday while I sleep). All I asked is that you demonstrated you had investigated and learned meaningful information associated with four tasks. The following is a combined document of the types of answers I was looking for:

Task One

Demonstrate that you understand XQuery via demonstration of meaningful queries against our class XML Database.

Your Answer:

XQuery provides aggregating functions to calculate quick mathematical values like average for a set of XML elements such as
snowwater equivalent in this example here. We limit the average calculation to those areas of study that actually contain 
snow cover. The query finds an average snow water cover on April 4th of .41 cubic meters in the Snowqualmie watershed.

let $x := avg(//data/location/values[snowwater>0]/snowwater)
return $x

Result: 

1)	0.41270174078510174

We want to know if a day was rainy or not. A rainy day is a day when the precipitation was higher than the 
average for the whole period under study. The following query replies with a single text response as to whether
April 4th was rainy or not (above or below average):

let $average := avg(//data/location/values/precip)
let $today := //data/location[x=3 and y=4]
  return
    if ($today > $average) then
      <data>
        The day { //data/date } was rainy.
      </data>
    else  
      <data>
        The day { //data/date } was OK.
      </data>

Returns:

<data>
The day 04-09-2006 was rainy.
</data>

XQuery functions can be used to do some processing before returning data. The query below creates a function that accepts 5 parameters: time 
interval, X coordinates (min and max), and Y coordinates (min and max). It then returns the average ocean temperature for that grid,
the area between X coordinates 25-27 and Y coordinates 26-29 at time period 1:

declare function local:avgOceanTemp(
  $atTime as xs:int?,
  $xMin   as xs:int?,
  $xMax   as xs:int?,
  $yMin   as xs:int?,
  $yMax   as xs:int?
)
    as xs:decimal?
{
let $loc := //data[time=$atTime and location/x>=$xMin and location/x<=$xMax and location/y>=$yMin and location/y<=$yMax]
let $tot := count($loc)

return  if ($tot>0)
        then <data>{sum($loc/location/values/oceantemp) div count($loc)}</data>
        else <data>0</data>
};

<average>{local:avgOceanTemp(1, 25, 27, 26, 29)}</average>

Returns:

<average>7.384105962238333</average>

This example query selects locations where the salinity is greater than 28.7 ppt and the zooplankton are greater than 0.07 mmo. It 
adds each returned location inside a root node called "featured-locations", extracting the required data. It also sorts these entries 
by zooplankton, salinity, and the respected coordinates. One of the great features of XQuery is that the return set can be embedded 
in XML. This allows users to have one of the best benefits of XML: humans can read and modify the data, but it is structured so that 
computers can, too. The query below will produce a document that could be fed automatically into a program that would examine the 
returned locations further.

<featured-locations>{
for $loc in //location[values/salinity > 28.7 and (values/zoo > 0.07)]
let $sal := $loc/values/salinity,
    $zoo := $loc/values/zoo,
    $x   := $loc/x,
    $y   := $loc/y
    order by $zoo descending, $sal descending, $x, $y
    return
    <location x="{data($x)}" y="{data($y)}">
       {$zoo}
       {$sal}
    </location>
}</featured-locations>

Returns:

<featured-locations>
	<location x ="50" y ="18">
		<zoo>0.07628769</zoo>
		<salinity>29.7909482017</salinity>
	</location>
	<location x ="47" y ="17">
		<zoo>0.07628769</zoo>
		<salinity>29.7592089519</salinity>
	</location>
	<location x ="36" y ="49">
		<zoo>0.07628769</zoo>
		<salinity>28.9168980911</salinity>
	</location>
	<location x ="37" y ="47">
		<zoo>0.07628769</zoo>
		<salinity>28.8241218224</salinity>
	</location>
</featured-locations>


Provide me with links to XQuery documents on the Web (tutorials, books, articles, etc.) that you used to mine for ideas and gain technical skills.

Your Answer:

Stylus Studio - Introduction to the XQuery FLWOR Expression
XQuery Lite
An Introduction to XQuery
Another Introduction to XQuery
2003 XQuery Tutorial
XQuery: Reinventing the Wheel?
O'Reily - What is XQuery
XQuery Studio
XQuery on Wikipedia
W3C XQuery Requirements
W3C XQuery Recommendation
Querying XML Data with XQuery
What is XQuery?
Oracle - XQuery Flowers
Oracle - XQuery: A New Way to Search
Querying, Constructing, and Transforming XML with Oracle XQuery
XQuery Presentation by CafeConLeche
W3 Schools XQuery Tutorial

Consider providing a paragraph or two about your opinions on XQuery usefulness and syntax.

Your Answer:

XQuery is a useful and effective way to gather relevant information from an XML database. The syntax is intuitive, especially the format used to reference nodes. The syntax reflects one of the intrinsic properties of XML: hierarchy. Compared to SQL, one can see that the FLWOR (For, Let, Where, Order by, Return) syntax is well equipped for drilling into a document, but is poorly equipped for handling entity relationships. Since relational databases are built around relational algebra, it makes sense that SQL implementations have syntax for joining and aggregating data.

Part of the usefulness of XML-the fact that it can be edited by humans, and read by machines-is also its downfall. Because every part of the XML document must be editable by humans without special tools, there is no way to efficiently search in a document. Because there's no mechanism to tell the computer at which byte of the document the desired node might be, the computer needs to store the entire XML document in memory when performing a query. This is acceptable for small amounts of data; but as we could see from the OWorld server, it's not very scalable. We ran eXist on a server with more memory than the OWorld server and only 80% of the class xml data, and could crash the server with one memory-intensive query. We did not find anything in the configuration files that suggested an alternative configuration to overcome the performance shortfall.

XML and XQuery seem to have an easier learning curve than relational databases and SQL. XML is extremely flexible and intuitive, so it definitely is appropriate to use. Because modeling entity relationships isn't native to XML and because of the performance limitations inherent to XML applications, those applications that require speedy processing of many data would be adversely affected by XML use. XML is a great format for subsets of all the data, partly because of the ease and intuitive nature of XQuery.

XQuery is useful for performing queries on embedded data in documents, applicable to database system such as catalogs and company records. By applying conditional expressions and comparisons, queries can also be customized to return context-sensitive searches. These advantages are highly beneficial for conducting searches in large relational databases. XQuery syntax takes the form of XML node (element, attributes, and variables) names, which integrates nicely with the growing popularity of XML, making XQuery a useful tool to use. It reads in XML fragments and returns a sequence in the same format, highly suitable as a data-oriented querying tool. It also generalizes existing SQL syntax with FLWOR (for-let-when-order-return) expressions, which can then be incorporated with HTML to generate customizable outputs in web browsers.

xQuery is a lot simpler language to learn than SQL. Conceptually the two technologies do the same thing: help you retrieve data. However xQuery seems like a video game where you dig for treasure with a map, navigating the various nodes to harvest what you need and then leave. xQuery is an easier concept to absorb for people getting introduced to both SQL and xQuery. However, SQL is more developed and it is easier to combine data from different tables using joins. xQuery is still a relatively young technology and probably will develop abilities to relate data better as time goes on. The same issues in SQL of having to understand the relationships between the data still exist in xQuery, and like in the construction of SQL great care should be taken in the design of XML documents and their DTDs.

Task Two

Turn in a list of locations each member in the group will be using for the Project 3 Web site. Each location should be justified by a short text explanation as to why that location was chosen. This list will help me write a script to get such data into the SQL database we'll use in Project 3.

Your Answer:

LocationLatitudeLongitudeExplanation
Mt. Rainier 46.85 N 121.75 W water bottling site
Possession Point Ferry 47.53.82' N 122.23.59' W good dive site
Edmonds Underwater Park47 48' 49.42'' N122 23' 01.46" W good dive site
Alki Seacrest Cove 247 35' 20.92" N122 22' 46.32" W good dive site
Golden Gardens/ Shilshole47 41' 25.98" N122 24' 22.6" W good dive site
Ft. Lewis Golf Course47.079 N122.726W balloon landing location
Snohomish Airport47.905223 N 122.105725W balloon landing location
Warren G. Magnuson Park47.6843 N 122.299W solar racing spot
Yukon Harbor 47.53134283040665 N122.5308609008789 W good dive site
Quarter-Master Harbor47.381149222795024 N122.46803283691406 W good dive site
Shilshole Bay47.67024262174783 N122.42408752441406 W good dive site
Sunset Beach47.346269 N122.556993 W good dive site
Fox Island East wall47.2315432 N 122.58819962 W good dive site
Harpers Ferry 47.5221536 N122.518802 Wgood dive site
Alki Beach 47.35 N -122.214 W frequented by divers
San Juan Island N.48.652N-123.045Wlarge mammals frequent
San Juan Island S.48.494N-122.958Wlarge mammals frequent
Snohomish River47.978N-122.185Wriver boat restoration
Everett47.979N-122.201Wcity guide
Seattle47.606N-122.331Wcity guide
Tacoma47.253N-122.443Wcity guide
Alki Beach Park 47.625N -122.263W good kite flying park
Marymoor Park 47.662N -122.127W good kite flying park
Gas Works Park 47.647N -122.336W good kite flying park
Magnuson Park 47.682N -122.263W good kite flying park
Blaine Elementary 47 38'11.28" N 122 24'32.23" W school used in site concept
UW Marine Science Bldg 47 38'58.10" N 122 18'45.62" W good field trip location
Alki Beach 47.58454202652335 N -122.4087334045425 W good surf location
Westport 46.88690348355735 N -124.1138897635217 W good surf location
Hole-In-The-Wall 48.38165632230724 N -124.7230996518806 W good surf location
Surf Site 47.70868725472133 N -122.3971208773579 W good surf location
Steven's Pass 47 50'46.09"N -121 40'09.03''W ski location
Crystal Mountain 47 12'28.58"N -121 59'16.58''W ski location
Mt. Baker 48 45'23.93"N -122 27'42.90''W ski location
Mohamed's House 47.66412 N -122.29348 W snow interest
Snohomish River 47 54'33.48N 122 05'37.5W river flow
Snohomish Basin 47 54'34.92N 122 05'33.9W soil state near river
Alki Beach 47 35'22.13N 122 23'37.46W good dive site
Hood Canal 47 49'09.70N 122 51'19.99W good dive site
Edmonds Underwater Park 47 49'00.32N 122 22'50.64W good dive site
Mukilteo 47 52'49.63N 122 20'05.24W good dive site
The Wynoochee River 47 37'48.70" N -123 61'40.44" W popular fly fishing
Snohomish river near Monroe 47 49'52" N 122 02'50" W pollution dumping history
Paradise, Mt Ranier 46 45' 31.27" N 122 02'03.74" mountain weather interest
Skagit River48 33' 26.51" N 121 24'45.44" W popular for recreational activities
Lake Ballinger park47.778N -122.327W golf course location
Mid-ocean Location48 23'31" N 123 60'50" W ideal for monitoring
Montlake Cut47.65 -122.316 closest water to UW core
Skykomish River 47.814 -121.578 interesting flow level
Puget Sound 47.66 -122.40 popular boating location
Bellevue 47.609 -122.146 location for sewage outlet
UW Campus 47.657 -122.312 weather for students
I-5 Seattle 47.679295 -122.3249420 localized weather report
Green Lake 47.68 -122.326 park setting

Task Three

Turn in one or more XSLT documents that I can use to convert an XQuery response set into an HTML page and view it in a Web browser.

Your Answer:

xmlxlsthtml
data.xmldata.xsldata.html
data.xmlsoil.xslsoil.html
data.xmlweather.xslweather.html

Consider writing a paragraph that explains how you could generate HTML output directly by using XQuery.

Your Answer:

XQuery returns valid XML and using the FLWOR methods we can choose to 
return the XML however we want. For example we can use our FLWOR methods 
to iterate over a result XQuery set and using the "return" method return 
the XML item wrapped in whatever HTML tags. Example: 

<html> 
<body> 
<ul> 
{ 
for $x in doc("books.xml")/bookstore/book/title 
order by $x 
return <li>{$x}</li> 
} 
</ul> 
</body> 
</html> 

*Taken from the W3Schools website: 
http://www.w3schools.com/xquery/xquery_flwor_html.asp

Task Four

Explore visualization techniques among your group that you would use to visualize data on your project 3 Web site.

Your Answer:

Map visualization

A simple way to create a visualization is to draw symbols for data categories on a geographical map of the area the data pertains to. An interactive map would allow a person to pick the variables they wanted to see on the right side and the following image would appear. The most that can be shown clearly is three variables at one time, more than that and the map gets messy. Data for this figure is totally fictional.

Bar Chart

Here is a stacked bar graph showing the relative contributions of 6 different locations to the total precipitation for a month. We used Matlab to generate the image. The location names would be labeled in the real thing. Data for this image was actually yearly data for 6 different years obtained at http://www.wrcc.dri.edu/cgi-bin/cliMONtpre.pl?waseat.

Line Chart

This figure plots lines from three variables on the same axis. It is set up as a prediction for the next 12 hours. We used Matlab to generate the image.

Profile plot

We found two simple ways to create graphics of profiles as long as you can get the data into an array/spreadsheet format. For the first method, we used Microsoft Excel and used the "Chart" function to make a data plot.

Cross section graphs

Using MatLab, we also found that you can use a variable vs. depth and visualize the ocean with using a color bar and have the colors represent different aspects of the using the "pcolor" function on MatLab. For example, we have various properties vs. depth shown here:

Try out some simple visualizations of XQuery outputs with a spreadsheet program. Provide me with links you find for interesting visualization techniques you would want to use if you only had the time and/or money to make it happen.