Geographic Information Systems (GIS) are by their nature data driven. The data comes in a wide variety of raster and vector formats. Rasters hold raw, continuous data recorded striaght from the real world. An example is Satellite/aerial imagery, this is a commonly held in an open format with broad support, such as GeoTIFF or GeoJPEG. Vector formats hold refined, discrete data, which has been manually traced or otherwise derived other data sources. Examples include building outlines, contours, road routes, pipe networks land land parcels and locations. Vector data is usually traced or derived, at great expense from raster data, to encode business information - as a result it’s usually highly valuable. Unfortunately, there are many GIS vector file formats, and most are proprietary. They can only be used to their full in their native software. Three of the biggest are AutoCAD DXF, MapInfo TAB and ArcGIS Personal Geodatabase. One vector format is unique - both an open standard, and in wide use: Shapefile Shapefile is publicly documented in ESRI Shapefile Technical Description by ESRI Inc., it’s creator. Any GIS software worth it’s salt can read and write to the format, so it’s become the least common denominator. It is the format for storing and exchanging vector data between teams, departments, businesses and government. In my opinion this makes Shapefile the best thing ever to happen to GIS, without it the GIS market would be a fraction of it’s current size. Despite it’s popularity, Shapefile does have some serious limitations, mainly due to it’s DBF heritage:»
Investigating the performance of an intranet mapping website this week, I was introduced MXDPERFSTAT. It’s a fantastic tool for investigating map display performance. Given an ArcMap document (a .mxd file) it runs ArcMap and loops over the map layers, displaying each at a list of scales (e.g. 1:100000, 1:10000, 1:2500, 1:500) automatically. Scale cut-offs are followed, in the same way ArcMap would. Once done, MXDPERFSTAT writes an html report, of the time taken and features retrieved to display each layer at each scale. Since it runs in situ, network delays and other real world bottlenecks are accounted for. It even highlights layers that are abnormally slow, or that fetch an excessive number of features. Perfect for diagnosing a slow map, or guiding a tune up of the infrastructure. Supporting ArcGIS installations, I’m a regular searcher on ArcScripts. How did I miss this for the last 4 years, and what other gems are there? What GIS utilities do you find indispensable?»
Sending a sensitive file, one that should be encrypted, amongst Linux and OSS geeks is doable. Most have heard of PGP, many have a GPG key (here is mine) and some even use it. Sending an encrypted file to most people is a non-starter. The software may be there (Outlook is S/MIME capable), but the knowledge and the experience definately isn’t. Which is a shame, because I’d like to have my bank statement securely sent to my email account. PGP Desktop has a feature called the Self Decrypting Archive. To quote the PGP Command Line for Servers FAQ:»
A couple of years ago I tried to use ArcObjects, through
It didn’t quite work. Last week I tried again, using the newly released
IronPython 2.0. This time it worked better.
is based on CreateSDEConnFile.java, from Creating ArcSDE connection
files on the fly using Python and
on ESRI’s Geoprocessing blog. For those not already familiar,
ArcGIS is by accounts the market
leader for Geographic Information System (GIS) software. The core of the
suite comprises ArcGIS Desktop, and ArcGIS Server. On the desktop ArcMap
is used to create map documents (.mxd file), whilst ArcCatalog is used
to manage data sources. ArcGIS Server can (amongst other things) serve a
map document, as a service for web client, Google Earth or remote ArcMap
users. ArcGIS may be automated to an extent, through an interface known
as ArcGIS Geoprocessing. But this covers only some cases, delving deeper
provides much greater opportunities. ArcGIS is built on a COM object
Native ArcGIS files, such as an ArcSDE connection (.sde file) are the in
memory COM object, serialised to disk as binary. It is difficult to edit
or create such files in an automated fashion, without calling
ArcObjects. So, like the Java code CreateSDEConnFile.py calls ArcObjects
directly. It can produce an ArcSDE connection file, suitable for
ArcCatalog. It works by calling the .NET bindings, through Interop
assemblies. Anything that can be done through VBA, or C# should be
possible through IronPython. There are a couple of rough edges.
ArcObjects is verbose, and IronPython requires some boilerplate to deal
with interfaces. Instead of writing
conn_props['SERVER'] = sys.argv
conn_props.SetProperty('SERVER', sys.argv) one needs to
esriSystem.IPropertySet.SetProperty(conn_props, 'SERVER', sys.argv)
This is explained properly in IronPython bug
To run the script call it as:
"c:Program FilesIronPythonipy.exe" create_sde_conn_file.py filename.sde hostname 5151 username password SDE.DEFAULT ""
My intention is to take this proof of concept further. To do the same
with layer files and map documents. Ultimately to create a build system,
able to automatically generate a complete ArcMap document, from textual
source files (e.g. json2mxd.py, mxd2xml.py). This would allow proper
version control of the source material, and automatic deployment of
ArcGIS Server map services. In the wider scheme, it should be possible
to create custom GIS applications with Python, using the full
capabilities ArcObjects and ArcGIS. Updated 29 Jan 2009: Added some
context, for those coming to this post from a Python background.
Here’s an email I received today:»
I have no evidence, and it’s wishful thinking more than anything. However, I predict that just before the expected release Microsoft will reveal Windows 7 is to be a free upgrade for Windows Vista users. P.S. If you have any trouble posting a comment to this blog, please let me know on email@example.com.»
Putting my timesheets in order today, I finally figured out how to make Excel deal correctly with time durations. The default is to treat values as a date/time, formatted as hh:mm. So a value such as 37:00 - meant as a duration - is displayed as 13:00 (1 PM the following day). To correct this, choose custom cell formatting, and enter the format as [h]:mm. In OpenOffice Calc, [H]:MM is the default format for a time value (tested with 3.0), so durations work out of the box. For something pre-cooked, the OpenOffice Documentation site has a timesheet template by Vivian Lal.»
I couldn’t find a Python cheat sheet that I liked, so I made another.»
This post is a tribute to Linux Hater who sadly has retired. He was insightful and right about many things, although sometimes a bit too whiny. Linux currently holds about 1% market share on the desktop. It has gained 0.5% in 2 years, whilst Mac OS X has gained 3%, and MS Windows has lost 4%. In the browser market Firefox is now nudging 20% market share. Can Linux ever achieve that? A single ‘Year of Linux on the desktop’ is unlikely, but it’s a popular meme, so lets to play. It’s some number of years in the future. Ubuntu ‘Satisfied Squirrel’ has built on slow, steady growth. Linux desktop share now nudges 20%. What might this future Linux desktop be like, compared to now?»
- The XBox 360 (Xenon), Wii (Broadway) and Playstation 3 (Cell) all use an IBM licensed PowerPC CPU.
- The Ubuntu spell-check dictionary includes ‘SPARCstation’, but not ‘PlayStation’.
- YSlow rocks.
- Users of IIS should not believe the official docs on HTTP Compression, instead go to Coding Horror.
- MythTV has a several second delay, changing between live channels, the delay is inherent to the architecture.
- FreeView are renumbering their channels and apparently removing friendly channel numbering from their broadcasts, in favour of firmware updates.
Backing up my home folder this weekend, in readiness for the Ubuntu Intrepid beta I spotted some unusual path names scroll by:»
It’s been Five years since I moved to Birmngham, to begin work with Defence Estates. I’ve enjoyed my time there, the people are fantastic and I’ve learnt much. It’s now time to move on; I want to find new challenges and broader horizons. If you’re looking for a DBA, a system administrator, someone skilled in GIS support, or in application support - then please email me or call me. My CV is online (as MS Word, OpenDocument, or PDF).»
In Fitts Law and Minimalism vs GTK+ and Qt I complained about the excessive use of borders and padding in GTK+ and Qt. Here’s what I’ve got so far (click for unscaled versions):»