From adoyle at eogeo.org Sat Mar 25 09:59:48 2006 From: adoyle at eogeo.org (Allan Doyle) Date: Sat Mar 25 09:59:54 2006 Subject: [tiling] worldKit meets OnEarth with tiling Message-ID: <4C574368-1877-432B-8EAD-AAB073F2C4A1@eogeo.org> Just when I was thinking it's time to give this list a poke, along comes Mikel with a great example of why tiling is a good thing. http://radio.weblogs.com/0100875/2006/03/25.html#a1233 At least I'm assuming that this uses a tiling scheme... Allan -- Allan Doyle +1.781.433.2695 adoyle@eogeo.org From mikel_maron at yahoo.com Sat Mar 25 12:07:07 2006 From: mikel_maron at yahoo.com (Mikel Maron) Date: Sat Mar 25 12:07:10 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <4C574368-1877-432B-8EAD-AAB073F2C4A1@eogeo.org> Message-ID: <20060325170707.12429.qmail@web30804.mail.mud.yahoo.com> Yes, the OnEarth Landsat cache does use a tiling scheme. Though it's slightly different from the usual, so I'll endeavor to explain this use case. Hopefully any generic tiling scheme description format can encompass this scheme, and it won't be too hard for other tools to implement. As normal, there is a power-of-two pyramid of tiles. Each tile is 512 x 512 pixels. Requests are made through the normal OnEarth WMS, and BBOXs corresponding to the tiling scheme are served from the cache. The images start from the northwest corner (90,-180) and are placed in non-overlapping tiles covering the world. At the lowest zoom level, each tile covers 256 degrees of latitude and longitude. So the global image is not divided in equal portions, and the tiles contain areas "off the map. The bounding boxes at the lowest zoom are -180,166,76,90 and 76,166,332,90 http://wms.jpl.nasa.gov/wms.cgi?request=GetMap&layers=global_mosaic&srs=EPSG:4326&format=image/jpeg&styles=&width=512&height=512&bbox=-180,166,76,90 http://wms.jpl.nasa.gov/wms.cgi?request=GetMap&layers=global_mosaic&srs=EPSG:4326&format=image/jpeg&styles=&width=512&height=512&bbox=76,166,332,90 At the next zoom level, each tile covers 128 degrees of latitude and longitude. The bounding boxes are [-180,38,-52,90] [-180,166,-52,38] [-52,38,76,90] [-52,166,76,38] [76,38,204,90] [76,166,204,38] The next zoom level is 64 degrees per tile, and so on. At 2 degrees per tile, the tiles no longer overlap the world boundary. The lowest resolution tiles are 2**-5 degrees per tile. Hopefully this makes sense -- it's pretty simple, probably easier to communicate with a couple pictures! How can we describe this and other use cases in a generic form? Is there any way to include this information in a GetCapabilities document? Mikel ----- Original Message ---- From: Allan Doyle To: tiling@lists.eogeo.org Sent: Saturday, March 25, 2006 2:59:48 PM Subject: [tiling] worldKit meets OnEarth with tiling Just when I was thinking it's time to give this list a poke, along comes Mikel with a great example of why tiling is a good thing. http://radio.weblogs.com/0100875/2006/03/25.html#a1233 At least I'm assuming that this uses a tiling scheme... Allan -- Allan Doyle +1.781.433.2695 adoyle@eogeo.org _______________________________________________ tiling mailing list tiling@lists.eogeo.org http://lists.eogeo.org/mailman/listinfo/tiling From adoyle at eogeo.org Sat Mar 25 12:22:36 2006 From: adoyle at eogeo.org (Allan Doyle) Date: Sat Mar 25 12:22:44 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <20060325170707.12429.qmail@web30804.mail.mud.yahoo.com> References: <20060325170707.12429.qmail@web30804.mail.mud.yahoo.com> Message-ID: <0AD5E89A-E620-46D9-8ECC-966AB40399FC@eogeo.org> On Mar 25, 2006, at 12:07, Mikel Maron wrote: > Yes, the OnEarth Landsat cache does use a tiling scheme. Though > it's slightly different from the usual, so I'll endeavor to explain > this use case. Hopefully any generic tiling scheme description > format can encompass this scheme, and it won't be too hard for > other tools to implement. > > > > As normal, there is a power-of-two pyramid of tiles. Each tile is > 512 x 512 pixels. Requests are made through the normal OnEarth WMS, > and BBOXs corresponding to the tiling scheme are served from the > cache. Yes, this is exactly the effect I envision being able to achieve with a standard scheme. > > The images start from the northwest corner (90,-180) and are placed > in non-overlapping tiles covering the world. At the lowest zoom > level, each tile covers 256 degrees of latitude and longitude. So > the global image is not divided in equal portions, and the tiles > contain areas "off the map. The bounding boxes at the lowest zoom > are -180,166,76,90 and 76,166,332,90 What do you think the benefits of this are? Can you easily deal with the date line and poles this way? > > http://wms.jpl.nasa.gov/wms.cgi? > request=GetMap&layers=global_mosaic&srs=EPSG:4326&format=image/ > jpeg&styles=&width=512&height=512&bbox=-180,166,76,90 > http://wms.jpl.nasa.gov/wms.cgi? > request=GetMap&layers=global_mosaic&srs=EPSG:4326&format=image/ > jpeg&styles=&width=512&height=512&bbox=76,166,332,90 > > At the next zoom level, each tile covers 128 degrees of latitude > and longitude. The bounding boxes are [-180,38,-52,90] > [-180,166,-52,38] [-52,38,76,90] [-52,166,76,38] [76,38,204,90] > [76,166,204,38] > > The next zoom level is 64 degrees per tile, and so on. At 2 degrees > per tile, the tiles no longer overlap the world boundary. The > lowest resolution tiles are 2**-5 degrees per tile. > > > Hopefully this makes sense -- it's pretty simple, probably easier > to communicate with a couple pictures! > > How can we describe this and other use cases in a generic form? Is > there any way to include this information in a GetCapabilities > document? That's my hope for an outcome developed by this list. Either a parameterized scheme that clients have to adapt to, or ideally a universal scheme that makes clients dead-simple. Allan > > Mikel > > > > ----- Original Message ---- > From: Allan Doyle > To: tiling@lists.eogeo.org > Sent: Saturday, March 25, 2006 2:59:48 PM > Subject: [tiling] worldKit meets OnEarth with tiling > > Just when I was thinking it's time to give this list a poke, along > comes Mikel with a great example of why tiling is a good thing. > > http://radio.weblogs.com/0100875/2006/03/25.html#a1233 > > At least I'm assuming that this uses a tiling scheme... > > Allan > > -- > Allan Doyle > +1.781.433.2695 > adoyle@eogeo.org > > > > _______________________________________________ > tiling mailing list > tiling@lists.eogeo.org > http://lists.eogeo.org/mailman/listinfo/tiling > > > -- Allan Doyle +1.781.433.2695 adoyle@eogeo.org From mikel_maron at yahoo.com Sat Mar 25 12:34:06 2006 From: mikel_maron at yahoo.com (Mikel Maron) Date: Sat Mar 25 12:34:08 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <0AD5E89A-E620-46D9-8ECC-966AB40399FC@eogeo.org> Message-ID: <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> > > The images start from the northwest corner (90,-180) and are placed > > in non-overlapping tiles covering the world. At the lowest zoom > > level, each tile covers 256 degrees of latitude and longitude. So > > the global image is not divided in equal portions, and the tiles > > contain areas "off the map. The bounding boxes at the lowest zoom > > are -180,166,76,90 and 76,166,332,90 > > What do you think the benefits of this are? Can you easily deal with > the date line and poles this way? This is the tiling scheme devised by the OnEarth administrator. I'm not sure of the advantage, it doesn't make it easier for requests -- I think it must be related to how he is storing the cache. I adapted worldKit to work with this existing scheme, it wasn't too tricky. Normally, worldKit tiles WMS requests in a more usual fashion. The lowest zoom level covers the entire world extent, with a power of two pyramid of tiles to some maximum zoom. I want worldKit to support any variations covered in the general scheme devised here. From adoyle at eogeo.org Sat Mar 25 12:56:34 2006 From: adoyle at eogeo.org (Allan Doyle) Date: Sat Mar 25 12:56:40 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> References: <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> Message-ID: On Mar 25, 2006, at 12:34, Mikel Maron wrote: >>> The images start from the northwest corner (90,-180) and are placed >>> in non-overlapping tiles covering the world. At the lowest zoom >>> level, each tile covers 256 degrees of latitude and longitude. So >>> the global image is not divided in equal portions, and the tiles >>> contain areas "off the map. The bounding boxes at the lowest zoom >>> are -180,166,76,90 and 76,166,332,90 >> >> What do you think the benefits of this are? Can you easily deal with >> the date line and poles this way? > > This is the tiling scheme devised by the OnEarth administrator. I'm > not sure of the advantage, it doesn't make it easier for requests > -- I think it must be related to how he is storing the cache. I > adapted worldKit to work with this existing scheme, it wasn't too > tricky. > > Normally, worldKit tiles WMS requests in a more usual fashion. The > lowest zoom level covers the entire world extent, with a power of > two pyramid of tiles to some maximum zoom. > > I want worldKit to support any variations covered in the general > scheme devised here. With the right scheme we can get at least two big wins: 1. Squid proxies or similar things would work well. If you could tell the proxy to only cache "standard" requests, then tiles would come from squid and special one-off requests would go to the server 2. standard tiles can be distributed. They can become disembodied from the server. Of course, #2 would imply that we have some way to figure out when things are stale. In #1, if the WMS behind the squid is properly HTTP aware, it can interact with the squid to deal with staleness. One way to deal with staleness is with a DNS-like TTL scheme. Every tile can have a time stamp and if it passes a certain age, it is considered stale. Different datasets can have different TTLs. -- Allan Doyle +1.781.433.2695 adoyle@eogeo.org From adam.hill at gmail.com Sun Mar 26 00:53:06 2006 From: adam.hill at gmail.com (Adam Hill) Date: Sun Mar 26 00:53:10 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> References: <0AD5E89A-E620-46D9-8ECC-966AB40399FC@eogeo.org> <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> Message-ID: This is actually the tiling scheme used by World Wind and Lucian implemented it on the server side so we would not melt down OnEarthm since it was our main source of high res data in the beginning. Unfortunately it wasnt enough and we got cut off :) We have a copy of the code he uses(d) at - http://www.twobeds.com/upload/userfiles/adamhill/lucians%20code%20-%20cache_1_1.gz if anyone is interested. So now the NASA WW groups hosts all the major datasets pretiled, but has also started using a 'heatmap' of tile request that they have captured over time and use that to drive the initial tiling, then do a just in time tiling if a tile is requested and has not been done yet. We also have some code that does the inverse, serve up WMS requests from the pretiled cache - http://www.twobeds.com/upload/userfiles/adamhill/From%20WMS%20Request%20to%20WW%20Tile%20Cache_php.txt. adam... The Free Earth Foundation From adam.hill at gmail.com Sun Mar 26 00:59:41 2006 From: adam.hill at gmail.com (Adam Hill) Date: Sun Mar 26 00:59:45 2006 Subject: [tiling] worldKit meets OnEarth with tiling In-Reply-To: <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> References: <0AD5E89A-E620-46D9-8ECC-966AB40399FC@eogeo.org> <20060325173406.56699.qmail@web30813.mail.mud.yahoo.com> Message-ID: This is actually the tiling scheme used by World Wind and Lucian implemented it on the server side so we would not melt down OnEarthm since it was our main source of high res data in the beginning. Unfortunately it wasnt enough and we got cut off :) We have a copy of the code he uses(d) at - http://www.twobeds.com/upload/userfiles/adamhill/lucians%20code%20-%20cache_1_1.gz if anyone is interested. So now the NASA WW groups hosts all the major datasets pretiled, but has also started using a 'heatmap' of tile request that they have captured over time and use that to drive the initial tiling, then do a just in time tiling if a tile is requested and has not been done yet. We also have some code that does the inverse, serve up WMS requests from the pretiled cache - http://www.twobeds.com/upload/userfiles/adamhill/From%20WMS%20Request%20to%20WW%20Tile%20Cache_php.txt. adam... Free Earth Foundation