Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create air quality dataset #2

Open
jimjonesbr opened this issue Jun 5, 2013 · 44 comments
Open

Create air quality dataset #2

jimjonesbr opened this issue Jun 5, 2013 · 44 comments

Comments

@jimjonesbr
Copy link
Member

Create air quality dataset (NOAA) for the demonstration at INSPIRE.

@ghost ghost assigned albhasan Jun 5, 2013
@mpfeil
Copy link
Contributor

mpfeil commented Jun 5, 2013

Simon mentioned PEGELONLINE. Not air quality but maybe interesting.
http://www.pegelonline.wsv.de/gast/start

@albhasan
Copy link
Contributor

albhasan commented Jun 6, 2013

I found a WMS, WFS but not a SOS. Can you help me finding the website's SOS? My German is not that good.

@mpfeil
Copy link
Contributor

mpfeil commented Jun 6, 2013

For PEGELONLINE SOS i found this link: http://pegelonline.wsv.de/webservices/gis/sos?request=GetCapabilities&service=SOS

@albhasan
Copy link
Contributor

@mpfeil : The triple store has more than 1 million triples now, all of them comming from 52 North's demo SOS service. Can you check how they look in the viewer? We need to select a good-looking time-window to make a video or to make a live demo.

Cheers!

@mpfeil
Copy link
Contributor

mpfeil commented Jun 11, 2013

@albhasan : Thanks.
I am testing right now. What looks a bit strange to me is if I query our SPARQL Endpoint

  1. getting the graphs is very very slow
  2. if I select the watertemperature property the "listFeaturesOfInterest" function of the facade returns 22 times the same FOI

@mpfeil
Copy link
Contributor

mpfeil commented Jun 11, 2013

  1. the "getFOILastObservation" of the facade is not responding anymore :(

@albhasan
Copy link
Contributor

Hi: Jim fix the getting graphs, it's way faster now

@jimjonesbr
Copy link
Member Author

Listing graphs fixed and repository updated. Now the listing graphs is quick enough 👍

@jimjonesbr
Copy link
Member Author

@mpfeil I just tested the listFeatureOfInterest method and it works perfectly. Could it be you're not iterating through the list properly? The query's resultset does not repeat any value

@mpfeil
Copy link
Contributor

mpfeil commented Jun 11, 2013

Okay. I will have a look at it. So the watertemperature property has one foi right?

@jimjonesbr
Copy link
Member Author

@mpfeil my apologies, it was my mistake. We never had a property with more than one foi, that's why I never noticed the a variable was outside the iteration loop.

It now should bring this result:

http://ifgi.uni-muenster.de/hydrolod#Nalje_Siel_126001
http://ifgi.uni-muenster.de/hydrolod#Obernau_24700302
http://ifgi.uni-muenster.de/hydrolod#Kleinheubach_24700200
http://ifgi.uni-muenster.de/hydrolod#Schwabelweis_10062000
http://ifgi.uni-muenster.de/hydrolod#Pfelling_10078000
http://ifgi.uni-muenster.de/hydrolod#Passau_Donau_10091008
http://ifgi.uni-muenster.de/hydrolod#Hofkirchen_10088003
http://ifgi.uni-muenster.de/hydrolod#Schweinfurt_Neuer_Hafen_24300304
http://ifgi.uni-muenster.de/hydrolod#Straubing_10074009
http://ifgi.uni-muenster.de/hydrolod#Faulbach_24700109
http://ifgi.uni-muenster.de/hydrolod#Astheim_24300406
http://ifgi.uni-muenster.de/hydrolod#Dandlbachmuendung_10098007
http://ifgi.uni-muenster.de/hydrolod#Raunheim_24900108
http://ifgi.uni-muenster.de/hydrolod#Deggendorf_10081004
http://ifgi.uni-muenster.de/hydrolod#Oberndorf_10056302
http://ifgi.uni-muenster.de/hydrolod#Mainflingen_24700325
http://ifgi.uni-muenster.de/hydrolod#Wuerzburg_24300600
http://ifgi.uni-muenster.de/hydrolod#Wertheim_24709089
http://ifgi.uni-muenster.de/hydrolod#Bamberg_24300042
http://ifgi.uni-muenster.de/hydrolod#Vilshofen_10089006
http://ifgi.uni-muenster.de/hydrolod#Steinbach_24500100
http://ifgi.uni-muenster.de/hydrolod#Frankfurt_Osthafen_24700404

I'm going to commit the change now.

@jimjonesbr
Copy link
Member Author

Commited. 👍

@mpfeil
Copy link
Contributor

mpfeil commented Jun 11, 2013

okay i will check it out and will look at it

@jimjonesbr
Copy link
Member Author

great! let me know when you can see them :)

@robarto
Copy link

robarto commented Jun 11, 2013

I big that from Wuppertal Oberbarmen

On Tue, Jun 11, 2013 at 9:51 PM, Jim Jones [email protected] wrote:

great! let me know when you can see them :)


Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-19287815
.

Thomas Bartoschek
ifgi - Institute for Geoinformatics
University of Münster
Weselerstr. 253
48151 Münster, Germany

Phone: +49 (0)251 83 30011
Mobile: +49 (0)173 511 6094
Fax: +49 (0)251 83 39763

Email: [email protected]
http://ifgi.uni-muenster.de/
http://www.gi-at-school.de/
http://www.uni-muenster.de/MExLab/
http://vespucci.org/

@jimjonesbr
Copy link
Member Author

@mpfeil , the last query (actually the most imporant one :D) is still taking too long, but @albhasan is working on it. If we don't solve it quickly, I would really suggest to drastically reduce the amount of triples for the sake of the presentation.

@robarto I hope you're not the one who is driving :) I big that from the empty ifgi 👍
Have a nice trip, greetz to uncle Sam !

@mpfeil
Copy link
Contributor

mpfeil commented Jun 11, 2013

Okay. I got it run. With the last one you mean getFOILastObservation?
The getGraph is really fast 👍

@jimjonesbr
Copy link
Member Author

With the last one you mean getFOILastObservation?

Exactly. Also the getObservationTimeInterval is probably affected.

@albhasan
Copy link
Contributor

I just deleted the triple store (it was broken anyway). This time I uploaded some properties just for a week instead of a month. The new triples agree with out new ontology interpretation but so far I'm not sure if they're "faster" to retrieve. Only the new queries will tell

@jimjonesbr
Copy link
Member Author

Nice! 👍
Can you adapt the queries to the new triples? I can then immediately update the factory!

@albhasan
Copy link
Contributor

I'll do it later, I'll let you know. I'm not done with all the variables yet. I just uploaded some partial results for you to be able to keep working without breaking the server (I wish!)

It's never that easy

@albhasan
Copy link
Contributor

@jimjonesbr : Can you please run some XSLTs in you computed tonight while you sleep? The new modifications implied a lot more of processing

@jimjonesbr
Copy link
Member Author

absolutely! Just give me the instructions and I will let it running :-)

@albhasan
Copy link
Contributor

Hi.

0 - Make sure your computer will not suspend or hybernate.
1 - Close other applications
2 - In dropbox, there is a file called "oneweek.7z". Download it and unzip
it.
3 - In the "oneWeek" folder, there is a file called "europaXSLT_JIM.txt".
Please open it, change the path accordingly and make it a bash.
4 - Run it from the terminal
5 - Wait for 10 minutes, if you got an error please give me a call

I was thinking that maybe we can make a bash from the whole process, I
mean, go to the SOS, store the observations, transform them and upload them
to the triple store.

Thanks, cheers!

2013/6/13 Jim Jones [email protected]

absolutely! Just give me the instructions and I will let it running :-)


Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-19407378
.

@albhasan
Copy link
Contributor

Hi: I updated the query "geFOILastObservation" and it's improved a lot.

@jimjonesbr You were right, the new ontology implementation made a huge difference. Can you please update the remainng query? I don't remember its name

@jimjonesbr
Copy link
Member Author

Awesome! I just tested the query and the application no longer freezes :-)
However, the globe does not show anything.

@mpfeil can you take a look? This is the error:

Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: 0 >= 0

Thanks, @albhasan !!

@mpfeil
Copy link
Contributor

mpfeil commented Jun 13, 2013

Wow it is really quick.
@jimjonesbr this is my error: Exception in thread "AWT-EventQueue-0" com.hp.hpl.jena.query.QueryParseException: Encountered "" at line 1, column 2502.
Was expecting one of:
...
<PNAME_NS> ...
<PNAME_LN> ...
<BLANK_NODE_LABEL> ...
...
...
"select" ...
"values" ...
"graph" ...
"optional" ...
"minus" ...
"bind" ...
"service" ...
"filter" ...
"true" ...
"false" ...
...
...
...
<INTEGER_POSITIVE> ...
<DECIMAL_POSITIVE> ...
<DOUBLE_POSITIVE> ...
<INTEGER_NEGATIVE> ...
<DECIMAL_NEGATIVE> ...
<DOUBLE_NEGATIVE> ...
<STRING_LITERAL1> ...
<STRING_LITERAL2> ...
<STRING_LITERAL_LONG1> ...
<STRING_LITERAL_LONG2> ...
"(" ...
...
"{" ...
"}" ...
"[" ...
...

This happens if I select a FOI and the error is called when I try to get the last FOI observation via the facade.

@jimjonesbr
Copy link
Member Author

mm... it seems there is a problem in the query :(
@albhasan do you have any idea?

@jimjonesbr
Copy link
Member Author

@mpfeil I just corrected the query and committed the bug fix, now the getFOILastObservation works.

@albhasan
Copy link
Contributor

Sorry guys, I guess I missed something in the query. Anyway, I was wondering about the units of measurement, I mean, the query for the latest observation retrieves just a number, should we show the user the units as well? Cheers!

@jimjonesbr
Copy link
Member Author

@albhasan good idea! I didn't notice it was missing :) Let's add it..

By the way, the XSLT transformation is still running. I guess it started about 10:00 ago... shall I still wait a bit longer?

@jimjonesbr
Copy link
Member Author

@albhasan I had to stop the process, I ran the script in console mode (no GUI) for 13:00h straight and it seemed to still be running. Sorry.

@albhasan
Copy link
Contributor

No problem! We need to do what we can with what we have

2013/6/14 Jim Jones [email protected]

@albhasan https://github.com/albhasan I had to stop the process, I ran
the script in console mode (no GUI) for 13:00h straight and it seemed to
still be running. Sorry.


Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-19449603
.

@jimjonesbr
Copy link
Member Author

It'd be good if you could take a look on the files.. the folder with the output files is now ~300MB

@albhasan
Copy link
Contributor

What do you mean?

@albhasan
Copy link
Contributor

@mpfeil : The test dataset we have now goes from the 2013-06-06 to 2013-06-12. So, the data covers partially the flooding, can you see it in the app?

@mpfeil
Copy link
Contributor

mpfeil commented Jun 18, 2013

i will check it when i am back at ifgi

@mpfeil
Copy link
Contributor

mpfeil commented Jun 18, 2013

@albhasan i looked through our datasets and right it does not contain the waterlevel set which Arne mentioned today. Can you add it to our triplestore so that we can show the flooding?

here is the link to the 52N platform from today´s morning: http://sensorweb.demo.52north.org/sensorwebclient-webapp-stable/

@albhasan
Copy link
Contributor

Hi:

I'm worried because I uploaded all the dataset we were able to transform, so probably that dataset was between the big ones we couldn't handle, even using Jim's machine.

@albhasan
Copy link
Contributor

Hi:

@mpfeil Can you please test the app with this graph http://ifgi.uni-muenster.de/testNOAA/? - Thanks

@mpfeil
Copy link
Contributor

mpfeil commented Jun 19, 2013

i tried it and i get for every available property fois. 👍

@mpfeil
Copy link
Contributor

mpfeil commented Jun 19, 2013

both opengis.net properties should not work right?

@mpfeil
Copy link
Contributor

mpfeil commented Jun 19, 2013

and for the mmisw foi is no SOSvalue available and the foi has no defaultgeometry

@mpfeil
Copy link
Contributor

mpfeil commented Jun 19, 2013

hey while i was testing i found a strange thing. i was selecting the property lufttemperatur and got 33 fois for it.
i selected the foi Bramsche_xxxxxx. And nothing happend.

By debugging I found out that Bramsche has no observations for lufttemperatur. It only has Chlorid and Windrichtung and Windgeschwingkeit if I try to get the last observation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants