From adam.hill at gmail.com  Thu Jan 26 11:40:51 2006
From: adam.hill at gmail.com (Adam Hill)
Date: Thu Jan 26 12:51:21 2006
Subject: [tiling] Example Zero
Message-ID: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>

To get the ball rolling - Here is a PDF that describes how NASA
WorldWind / Punt does tiling currently.

http://www.ceteranet.com/nww-tile-struct.pdf

In the configuration data on the client side for a given Layer we have
a NumberOfLevels, LevelZeroTIleSize Degrees and TileSize. Its just a
simple powers of 2 system.

This is currently being served for the WW community by:
1) NASA using just a filesystem backend for almost all of WW's
datasets, including elevation.
2) Terraserver, which pulls the tiles from a SQL database
3) OnEarth with a WMS + a smart CGI script + a cache for WW, written
by Lucian Plesea.
4) The Free Earth Foundation server for the Zoomit Dataset (Robbin
Island, Mass.gov hires and some other hi-res but small datasets) and
others using a packed tile scheme (pyramids) and some simple PHP.

All of the above respond to 
"<URL>/tileset?L=<level>&X=<x-grid-coord>&Y=<y-grid-coord>" HTTP
GET's.

The mysteries to me are - picking optimal LZTSD (can one size really
fit all?) and outside of how much bandwidth it consumes is a tile size
of 512 really so much better/worse than 128 or even 1024?
From osgis.lists at gmail.com  Thu Jan 26 15:34:01 2006
From: osgis.lists at gmail.com (David Bitner)
Date: Thu Jan 26 15:34:04 2006
Subject: [tiling] Example Zero
In-Reply-To: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
Message-ID: <71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>

One thing that I'd like to get set up is a cascading WMS that I can
put at my office that would take regular WMS requests and would then
split the request into tiles, cache the tiles locally, and then sew
the tiles back together to deliver to my client applications whether
it be a desktop WMS client or MapServer or whatver.

With this type of architecture, There wouldn't necessarily have to be
any change in the original WMS servers or to any of my client
applications.  It would just act as a smart cache for data.  If this
was to use a "standardized request" and a WMS server that it hit was
already tiled to that dimension, you could get additional
opportunities to cache there.

On 1/26/06, Adam Hill <adam.hill@gmail.com> wrote:
> To get the ball rolling - Here is a PDF that describes how NASA
> WorldWind / Punt does tiling currently.
>
> http://www.ceteranet.com/nww-tile-struct.pdf
>
> In the configuration data on the client side for a given Layer we have
> a NumberOfLevels, LevelZeroTIleSize Degrees and TileSize. Its just a
> simple powers of 2 system.
>
> This is currently being served for the WW community by:
> 1) NASA using just a filesystem backend for almost all of WW's
> datasets, including elevation.
> 2) Terraserver, which pulls the tiles from a SQL database
> 3) OnEarth with a WMS + a smart CGI script + a cache for WW, written
> by Lucian Plesea.
> 4) The Free Earth Foundation server for the Zoomit Dataset (Robbin
> Island, Mass.gov hires and some other hi-res but small datasets) and
> others using a packed tile scheme (pyramids) and some simple PHP.
>
> All of the above respond to
> "<URL>/tileset?L=<level>&X=<x-grid-coord>&Y=<y-grid-coord>" HTTP
> GET's.
>
> The mysteries to me are - picking optimal LZTSD (can one size really
> fit all?) and outside of how much bandwidth it consumes is a tile size
> of 512 really so much better/worse than 128 or even 1024?
> _______________________________________________
> tiling mailing list
> tiling@lists.eogeo.org
> http://lists.eogeo.org/mailman/listinfo/tiling
>
From osgis.lists at gmail.com  Thu Jan 26 16:42:52 2006
From: osgis.lists at gmail.com (David Bitner)
Date: Thu Jan 26 16:42:54 2006
Subject: [tiling] Example Zero
In-Reply-To: <d5ec7b310601261332p5b74fbaay6d48e477ebd9dc5@mail.gmail.com>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
	<d5ec7b310601261332p5b74fbaay6d48e477ebd9dc5@mail.gmail.com>
Message-ID: <71c3c6c50601261342m6ab26842u5a9cb9c414302409@mail.gmail.com>

Thanks, I'll take a look at these.

On 1/26/06, Adam Hill <adam.hill@gmail.com> wrote:
> That is what OnEarth kinda does, but instead of cascaded WMS it just uses a
> CGI to sniff a tile request (ie WorldWind) look in the cache for it and if
> not fetch it from WMS and then stuff the result in the cache. Here is a copy
> of Lucians code he posted a while back -
> http://www.twobeds.com/upload/userfiles/adamhill/lucians%20code%20-%20cache_1_1.gz
>
> More code to get the ball rolling:
>
> A small piece of code showing how you can easily convert a WMS request into
> a specific tile request -
> http://www.twobeds.com/upload/userfiles/adamhill/From%20WMS%20Request%20to%20WW%20Tile%20Cache_php.txt
>
> And last but not least. The person who tiles data for the FEF has written a
> C++ app that tiles up data using the GDAL + PROJ.4 toolchain and has
> released it under BSD.
> http://www.twobeds.com/upload/userfiles/adamhill/dstile-0_2.zip
>
> adam...
From adoyle at eogeo.org  Thu Jan 26 19:13:04 2006
From: adoyle at eogeo.org (Allan Doyle)
Date: Thu Jan 26 19:13:08 2006
Subject: [tiling] Example Zero
In-Reply-To: <71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
Message-ID: <4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>

Right. The ur-WMS generating the tiles need not be a "tiling" WMS.

This blog post points to some people at MassGIS that were using a  
regular WMS with Squid in front of it. They built a ka-map client  
that requested only well-defined tiles (or does ka-map do that  
anyway?). Once the squid gets loaded up, it serves the tiles, the WMS  
just loafs in the background.

I think the key behind this concept is to make clients that all ask  
for the same tile boundaries. That means that the calculation of  
"neighboring tile" should be very quick and easy, ideally simple math  
so the clients can remain lightweight.

If by some great coincidence, the WMS tiling happens to line up with  
WW tiling, then a simple perl, python, etc. script could translate  
from one to the other (in either direction) and act as a gateway.

	Allan

On Jan 26, 2006, at 15:34, David Bitner wrote:

> One thing that I'd like to get set up is a cascading WMS that I can
> put at my office that would take regular WMS requests and would then
> split the request into tiles, cache the tiles locally, and then sew
> the tiles back together to deliver to my client applications whether
> it be a desktop WMS client or MapServer or whatver.
>
> With this type of architecture, There wouldn't necessarily have to be
> any change in the original WMS servers or to any of my client
> applications.  It would just act as a smart cache for data.  If this
> was to use a "standardized request" and a WMS server that it hit was
> already tiled to that dimension, you could get additional
> opportunities to cache there.
>
> On 1/26/06, Adam Hill <adam.hill@gmail.com> wrote:
>> To get the ball rolling - Here is a PDF that describes how NASA
>> WorldWind / Punt does tiling currently.
>>
>> http://www.ceteranet.com/nww-tile-struct.pdf
>>
>> In the configuration data on the client side for a given Layer we  
>> have
>> a NumberOfLevels, LevelZeroTIleSize Degrees and TileSize. Its just a
>> simple powers of 2 system.
>>
>> This is currently being served for the WW community by:
>> 1) NASA using just a filesystem backend for almost all of WW's
>> datasets, including elevation.
>> 2) Terraserver, which pulls the tiles from a SQL database
>> 3) OnEarth with a WMS + a smart CGI script + a cache for WW, written
>> by Lucian Plesea.
>> 4) The Free Earth Foundation server for the Zoomit Dataset (Robbin
>> Island, Mass.gov hires and some other hi-res but small datasets) and
>> others using a packed tile scheme (pyramids) and some simple PHP.
>>
>> All of the above respond to
>> "<URL>/tileset?L=<level>&X=<x-grid-coord>&Y=<y-grid-coord>" HTTP
>> GET's.
>>
>> The mysteries to me are - picking optimal LZTSD (can one size really
>> fit all?) and outside of how much bandwidth it consumes is a tile  
>> size
>> of 512 really so much better/worse than 128 or even 1024?
>> _______________________________________________
>> tiling mailing list
>> tiling@lists.eogeo.org
>> http://lists.eogeo.org/mailman/listinfo/tiling
>>
> _______________________________________________
> tiling mailing list
> tiling@lists.eogeo.org
> http://lists.eogeo.org/mailman/listinfo/tiling
>

-- 
Allan Doyle
+1.781.433.2695
adoyle@eogeo.org


From adoyle at eogeo.org  Thu Jan 26 19:18:58 2006
From: adoyle at eogeo.org (Allan Doyle)
Date: Thu Jan 26 19:19:02 2006
Subject: [tiling] Example Zero
In-Reply-To: <4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
	<4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>
Message-ID: <FC143703-5D44-4999-8133-0660203D36B7@eogeo.org>

Forgot the URL - http://think.random-stuff.org/FrontPage/archive/ 
2005/11/16/gis-day-in-boston

On Jan 26, 2006, at 19:13, Allan Doyle wrote:

> Right. The ur-WMS generating the tiles need not be a "tiling" WMS.
>
> This blog post points to some people at MassGIS that were using a  
> regular WMS with Squid in front of it. They built a ka-map client  
> that requested only well-defined tiles (or does ka-map do that  
> anyway?). Once the squid gets loaded up, it serves the tiles, the  
> WMS just loafs in the background.
>
> I think the key behind this concept is to make clients that all ask  
> for the same tile boundaries. That means that the calculation of  
> "neighboring tile" should be very quick and easy, ideally simple  
> math so the clients can remain lightweight.
>
> If by some great coincidence, the WMS tiling happens to line up  
> with WW tiling, then a simple perl, python, etc. script could  
> translate from one to the other (in either direction) and act as a  
> gateway.
>
> 	Allan
>
> On Jan 26, 2006, at 15:34, David Bitner wrote:
>
>> One thing that I'd like to get set up is a cascading WMS that I can
>> put at my office that would take regular WMS requests and would then
>> split the request into tiles, cache the tiles locally, and then sew
>> the tiles back together to deliver to my client applications whether
>> it be a desktop WMS client or MapServer or whatver.
>>
>> With this type of architecture, There wouldn't necessarily have to be
>> any change in the original WMS servers or to any of my client
>> applications.  It would just act as a smart cache for data.  If this
>> was to use a "standardized request" and a WMS server that it hit was
>> already tiled to that dimension, you could get additional
>> opportunities to cache there.
>>
>> On 1/26/06, Adam Hill <adam.hill@gmail.com> wrote:
>>> To get the ball rolling - Here is a PDF that describes how NASA
>>> WorldWind / Punt does tiling currently.
>>>
>>> http://www.ceteranet.com/nww-tile-struct.pdf
>>>
>>> In the configuration data on the client side for a given Layer we  
>>> have
>>> a NumberOfLevels, LevelZeroTIleSize Degrees and TileSize. Its just a
>>> simple powers of 2 system.
>>>
>>> This is currently being served for the WW community by:
>>> 1) NASA using just a filesystem backend for almost all of WW's
>>> datasets, including elevation.
>>> 2) Terraserver, which pulls the tiles from a SQL database
>>> 3) OnEarth with a WMS + a smart CGI script + a cache for WW, written
>>> by Lucian Plesea.
>>> 4) The Free Earth Foundation server for the Zoomit Dataset (Robbin
>>> Island, Mass.gov hires and some other hi-res but small datasets) and
>>> others using a packed tile scheme (pyramids) and some simple PHP.
>>>
>>> All of the above respond to
>>> "<URL>/tileset?L=<level>&X=<x-grid-coord>&Y=<y-grid-coord>" HTTP
>>> GET's.
>>>
>>> The mysteries to me are - picking optimal LZTSD (can one size really
>>> fit all?) and outside of how much bandwidth it consumes is a tile  
>>> size
>>> of 512 really so much better/worse than 128 or even 1024?
>>> _______________________________________________
>>> tiling mailing list
>>> tiling@lists.eogeo.org
>>> http://lists.eogeo.org/mailman/listinfo/tiling
>>>
>> _______________________________________________
>> tiling mailing list
>> tiling@lists.eogeo.org
>> http://lists.eogeo.org/mailman/listinfo/tiling
>>
>
> -- 
> Allan Doyle
> +1.781.433.2695
> adoyle@eogeo.org
>
>
> _______________________________________________
> tiling mailing list
> tiling@lists.eogeo.org
> http://lists.eogeo.org/mailman/listinfo/tiling
>

-- 
Allan Doyle
+1.781.433.2695
adoyle@eogeo.org


From adam.hill at gmail.com  Fri Jan 27 01:02:32 2006
From: adam.hill at gmail.com (Adam Hill)
Date: Fri Jan 27 08:25:40 2006
Subject: [tiling] Example Zero
In-Reply-To: <FC143703-5D44-4999-8133-0660203D36B7@eogeo.org>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
	<4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>
	<FC143703-5D44-4999-8133-0660203D36B7@eogeo.org>
Message-ID: <d5ec7b310601262202o409cafa7nfd0cbb360ff8115@mail.gmail.com>

But picking those standard numbers (or sets of numbers) for the geographic
slice and the tile size to get cache requests to line up between clients, is
where the devil is. Anyone know what the GMaps numbers are? Anyone have
suggestions for different numbers?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.eogeo.org/pipermail/tiling/attachments/20060127/3c2c6222/attachment.htm
From adoyle at eogeo.org  Fri Jan 27 08:38:58 2006
From: adoyle at eogeo.org (Allan Doyle)
Date: Fri Jan 27 08:39:02 2006
Subject: [tiling] Example Zero
In-Reply-To: <d5ec7b310601262202o409cafa7nfd0cbb360ff8115@mail.gmail.com>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
	<4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>
	<FC143703-5D44-4999-8133-0660203D36B7@eogeo.org>
	<d5ec7b310601262202o409cafa7nfd0cbb360ff8115@mail.gmail.com>
Message-ID: <0EA2766C-CDC1-4709-BC1C-47C07A1E15F6@eogeo.org>

Google has an "odd" projection. Or lets say I don't think they use  
4326. I think any scheme we come up with should allow for variations  
in projection and scheme but there needs to be some way to describe  
those schemes and ideally one of the schemes needs to "win" in the  
sense that most tiles are available in that scheme.

	Allan

On Jan 27, 2006, at 01:02, Adam Hill wrote:

> But picking those standard numbers (or sets of numbers) for the  
> geographic slice and the tile size to get cache requests to line up  
> between clients, is where the devil is. Anyone know what the GMaps  
> numbers are? Anyone have suggestions for different numbers?
> _______________________________________________
> tiling mailing list
> tiling@lists.eogeo.org
> http://lists.eogeo.org/mailman/listinfo/tiling

-- 
Allan Doyle
+1.781.433.2695
adoyle@eogeo.org


From adoyle at eogeo.org  Fri Jan 27 08:51:39 2006
From: adoyle at eogeo.org (Allan Doyle)
Date: Fri Jan 27 08:51:49 2006
Subject: [tiling] Example Zero
In-Reply-To: <0EA2766C-CDC1-4709-BC1C-47C07A1E15F6@eogeo.org>
References: <d5ec7b310601260840n7a4b881bu34cc4ec498a25fb7@mail.gmail.com>
	<71c3c6c50601261234p359b3da0v6875552957ded9d9@mail.gmail.com>
	<4886224B-CF5B-4FEF-8C69-CEA9D9F0ADA1@eogeo.org>
	<FC143703-5D44-4999-8133-0660203D36B7@eogeo.org>
	<d5ec7b310601262202o409cafa7nfd0cbb360ff8115@mail.gmail.com>
	<0EA2766C-CDC1-4709-BC1C-47C07A1E15F6@eogeo.org>
Message-ID: <9544E837-906B-401C-A33D-A9CA5167B5A4@eogeo.org>


On Jan 27, 2006, at 08:38, Allan Doyle wrote:

> Google has an "odd" projection. Or lets say I don't think they use  
> 4326. I think any scheme we come up with should allow for  
> variations in projection and scheme but there needs to be some way  
> to describe those schemes and ideally one of the schemes needs to  
> "win" in the sense that most tiles are available in that scheme.

That being said, I think a "proof of concept" scheme to show how it  
would all work is not a bad approach, rather than trying to go  
straight to the most flexible one.

Another tiling scheme is the osgPlanet scheme, I'm not sure where  
it's documented.

	Allan

>
> 	Allan
>
> On Jan 27, 2006, at 01:02, Adam Hill wrote:
>
>> But picking those standard numbers (or sets of numbers) for the  
>> geographic slice and the tile size to get cache requests to line  
>> up between clients, is where the devil is. Anyone know what the  
>> GMaps numbers are? Anyone have suggestions for different numbers?
>> _______________________________________________
>> tiling mailing list
>> tiling@lists.eogeo.org
>> http://lists.eogeo.org/mailman/listinfo/tiling
>
> -- 
> Allan Doyle
> +1.781.433.2695
> adoyle@eogeo.org
>
>

-- 
Allan Doyle
+1.781.433.2695
adoyle@eogeo.org


From mikel_maron at yahoo.com  Fri Jan 27 09:00:54 2006
From: mikel_maron at yahoo.com (Mikel Maron)
Date: Fri Jan 27 09:01:02 2006
Subject: [tiling] Example Zero
In-Reply-To: <0EA2766C-CDC1-4709-BC1C-47C07A1E15F6@eogeo.org>
Message-ID: <20060127140054.96156.qmail@web31811.mail.mud.yahoo.com>

>  I think any scheme we come up with should allow for variations  
> in projection and scheme but there needs to be some way to describe  
> those schemes and ideally one of the schemes needs to "win" in the  
> sense that most tiles are available in that scheme.

I agree. It's much easier for WMS clients to adjust how they request tiles than to get all WMS servers to agree on a common optimal scheme at the outset.

For instance, the OnEarth Landsat WMS has an optimized caching -- the lowest resolution level has .5 degrees per pixel, is 512 x 512, and the tiles are placed with no overlap. So at a worldwide scale, there are two tiles, one with a bbox of [-180,166,76,-90] and another with a bbox of [76,166,332,-90].  Pixels lying outside the real extent of the earth are simply clipped on the client side. A pyramid of tiles is formed with powers of two. The wms configuration in worldKit allows for options fitting this scheme, or any variations for other wms servers.

There's then a need for some simple format for a WMS server to specify it's preferred tiling settings. Perhaps some small XML format? Is there any WMS "request" type that can be used for this purpose?

Mikel



From nhv at cape.com  Fri Jan 27 09:01:12 2006
From: nhv at cape.com (Norman Vine)
Date: Fri Jan 27 09:01:16 2006
Subject: [tiling] Example Zero
In-Reply-To: <9544E837-906B-401C-A33D-A9CA5167B5A4@eogeo.org>
Message-ID: <EKEJIKAILPONGGENMBGAOEIIOOAA.nhv@cape.com>

Allan Doyle writes:
> 
> Another tiling scheme is the osgPlanet scheme, I'm not sure where  
> it's documented.

osgPlanet uses the 2 hemispheres ( East West ) for level 0 tiles
eg 180 x 180 degree tiles

then uses a recursive quadtree subdivision from there

This is a "natural" scheme for global data sets when using
a simple lat lon projection eg ESPG:4326 and easily 
implemented with just a little bit shifting

What I would really like to see is a method whereby servers
could advertise their tiling scheme something like the WMS
GetCapabilities request then clients could adapt accordingly

Glad to see this topic getting some attention finally

FYI  I will be offline till early next week 

Norman